[Telepathy] telepathy-spec process and D-Bus tools improvements, revisited

[Simon McVittie]

As part of some work on the connection-manager library I've been
revisiting the Telepathy specification format, in order to do
automatic generation of GInterfaces and so on.

The format the spec is written in is now documented,
and can be found on <http://telepathy.freedesktop.org/wiki/DbusSpec>.
I hope I can basically freeze this format for a while.

I've tried to make it all language-agnostic (most names are in the form
Camel_Case_With_Underscores because that's easy to map to any of
lower_case, UPPER_CASE or CamelCase using tr(1) or XSLT's translate(), for
instance), and make names consistent so they don't have to be
nearly-duplicated for lots of minor variations. If anyone has suggestions for
other improvements, now would be a great time to make them.

I want to improve the situation regarding auto-generated code to
avoid the situation currently present in Gabble, where we have generated
template files which are edited to become the actual source; the
template files are occasionally re-generated and the changes merged into
the source. This is a bit mad.

At the moment in my refactoring efforts (look for the various "gabble-cmlib"
branches on http://projects.collabora.co.uk/~daf/monkey), Gabble's lib
directory (which will eventually become the connection-manager library, and
possibly a libtelepathy replacement too) contains around a megabyte of
generated source checked-in to Darcs (with a Makefile target that
updates it from the spec directory). This is also a bit mad, but at
least each file is either generated or hand-written, never both.

My plan is that whenever something is auto-generated from the spec, its
source tarball will contain a copy of the specification it's produced
from, and either the tools used for generation or (as our toolchain
stabilises, and perhaps even becomes used outside Telepathy) a description
of where to get those tools. The generated files will not be distributed in
tarballs; they'll be generated at build time using the specification copy
as source.

When the specification changes, connection managers, the connection
manager library and so on will need updating anyway. The update process
will be to replace the spec copy with a newer one, update the tools if
the spec format has changed (I hope not), and fix everything in the
source that was broken by the change. This is something that has to be
done manually, perhaps with script assistance - that's fine, we can't
avoid that.

The connection manager library and its GInterfaces raise some interesting
issues regarding ABI, API and when it's acceptable to make what change.
The spec will end up producing three "interfaces" (using the word in a
loose sense):

 * the C API and ABI between connection-manager library and connection manager
 * the D-Bus interface between connection manager and client
 * the C API and ABI between client library and client

When telepathy-python and other bindings are updated to use more
generated code, they can be substituted for "C" in the above.

At the moment adding a method to an interface breaks the C ABI of the
connection-manager library (it makes the GObjectClass larger by adding
another function pointer), and even if that wasn't the case, calling
that method would cause a segfault in existing CMs because they don't
provide an implementation.

I'll have to change that so the ABI is number- and order-independent
(name-based), and there's a default implementation for every method which
always returns the NotImplemented error if the CM doesn't provide an
implementation. The CM won't be able to implement the method if it doesn't
have a new enough library, and the client won't be able to call it if *it*
doesn't have a new enough library, but upgrading the library shouldn't break
anything too drastically.

I believe adding signals is always OK: it certainly ought to be. The CM
won't be able to emit the signal unless its library is new enough, and
clients might not be able to bind to the signal unless *their* library
is new enough, but updating the library should, again, not break
anything.

Removing methods or signals, or changing method parameters, will cause
an incompatible change to the library (for instance in C its SONAME will
have to increase). We'll have to think about how that works in
development-branch work - perhaps unstable copies of CMs should be
statically linked to a correspondingly patched CM library, or perhaps we
should just be more backwards-compatible in our spec changes :-)

	Simon