[RFC] dbus-python API (re)definition

[Simon McVittie]

As mentioned on #dbus, I plan to re-implement the dbus_bindings part of
dbus-python using the Python C API rather than Pyrex, and generally sort
out the dbus-python API. I'm also working on getting some API documentation
together, using epydoc and reStructuredText.

Issues I'm aware of at the moment are as follows. If some of these
already have a solution that I haven't noticed, or if you're aware of
other issues, please comment.

API vagueness
=============
Due to Python's "public by default" API visibility combined with some
use of "import *", it's not clear which names are meant to be public.
It's reasonably clear that "import dbus ; dbus.dbus.dbus.dbus.dbus"
is not meant to work, but currently it does!

I'm trying to add the magic __all__ attribute to each module - as well
as making "import *" import only those names listed in __all__, this
gives epydoc a hint that it should display documentation for those names
(and only those names) in the page for this module, even if they were
actually imported from elsewhere (for instance, SessionBus should be
documented in dbus rather than in dbus._dbus). The API documentation can
then be the canonical reference for what you should import from where.

I've looked through the packages in current Debian unstable that depend
on python-dbus (these are avahi-discover, service-discovery-applet,
gnome-osd, hal-device-manager, listen and smart-notifier), and
Collabora's Telepathy IM code, to get an idea of what library users think
the public API currently is.

Here's what I *think* the public API should look like in terms of what
you can rely on being able to import, and from where:

  from dbus import version
  # actually from dbus._dbus
  from dbus import Bus, SessionBus, SystemBus, StarterBus, Interface
  # actually from dbus_bindings via dbus.types
  from dbus import Byte, Boolean, \
                   Int16, UInt16, Int32, UInt32, Int64, UInt64, \
                   Double, String, Array, Struct, Dictionary, Variant
  # also actually from dbus_bindings via dbus.types
  from dbus import Signature, ObjectPath, ByteArray
  # actually from dbus.exceptions or dbus_bindings
  from dbus import DBusException, ConnectionError, \
                   MissingErrorHandlerException, NameExistsException, \
                   ValidationException, IntrospectionParserException, \
                   UnknownMethodException, MissingReplyHandlerException

  # renamed from the current dbus.service, see below
  from dbus.exports import Object, BusName
  # actually from dbus.decorators
  from dbus.exports import method, signal

  import dbus.glib

Some of the constants from dbus_bindings could probably do with being exposed
somewhere, perhaps in dbus.constants or directly in dbus.

Rob McQueen points out that dbus.service should be renamed to something
without "service" in its name, like dbus.exports, since the term
"service" is deprecated; in that case we should provide a wrapper
dbus.service which raises DeprecationWarning.

There should probably be some sort of API for watches so you can run
dbus-python without using the glib mainloop (perhaps in Twisted, Tkinter or
PyQT code). I'm not sure at this point whether Messages and PendingCalls
should be exposed to Python code or not - what functionality would we
lose if they aren't?

Other API use I noticed:

* service-discovery-applet uses dbus.dbus_bindings.DBusException, so
  I'll probably add a stub dbus.dbus_bindings which issues
  a DeprecationWarning on import, for backwards compatibility.

API I'd like to make retroactively non-public, deprecate or just remove:

* Importing anything from dbus.proxies, dbus.matchrules, dbus.decorators,
  dbus.introspect_parser (rename them to dbus._proxies etc), and perhaps
  also from dbus.types and dbus.exceptions - the implementations will
  of course stay, since some of the classes are visible when returned
  from public methods or by being imported into dbus and dbus.exports

* Anything mentioning dbus_bindings or dbus_glib_bindings

Disconnecting
=============

I hear rumours that the current signal code can't disconnect from
signals, and that the current object exporting can't "unexport" objects,
meaning the relevant Python object is kept alive forever. I haven't yet
dug far enough to find out whether these are true, or false but the API
is non-obvious. I intend to fix both of these somehow.

Exported objects without a well-known name
==========================================

It should be possible to instantiate an Object when the application has
only a unique name. At the moment, it isn't, as you have to pass a
BusName to the Object constructor; the Object only really needs a Bus.

Queueing for well-known names
=============================

It should be possible for an application to control its well-known name
registrations better than it does at the moment. This may mean writing a
replacement for BusName.

Proxies for well-known names vs proxies for unique names
========================================================

As discussed on this list in another context, it should be possible for
client code creating a proxy to specify whether it wants to resolve the
well-known name into a unique name immediately, to allow stateful
communication (as dbus-python currently does), or whether it wants to
have a proxy representing "whatever application currently owns
com.example.GenericTextEditor" which will raise exceptions from method
calls if no application currently owns that well-known name.

dbus_bindings vs dbus.dbus_bindings
===================================

dbus_bindings thinks its full canonical name is dbus_bindings, but it's
installed inside the dbus Python package, i.e. as "dbus.dbus_bindings".
This confuses introspection tools like pychecker and epydoc, which IMO
is reason enough to fix it!

Due to dbus.pth and the possible separation of arch-dependent and
arch-independent parts of dbus-python (e.g. between /usr/lib64 and
/usr/lib on certain AMD64 Linux distributions, its canonical name may
in fact be dbus_bindings after all, as the dbus directory may
simultaneously be part of the Python search path, and a subdirectory
of a directory on the search path.

This does not seem ideal.

Best practice for Python extensions appears to be to install them as
top-level modules. Since dbus_bindings is meant to be private, I propose
to rename it to _dbus_bindings as a top-level Python module (so it ends
up in /usr/lib/python2.4/site-packages/_dbus_bindings.so or whatever,
and you always import _dbus_bindings, never dbus._dbus_bindings).

Alternatively we could just not install it in the dbus package, so it
ends up as /usr/lib/python2.4/site-packages/dbus_bindings.so, but I
don't think we want to encourage people to use it.

Exactly the same applies to dbus_glib_bindings, just insert glib_ where
appropriate.


Comments?

        Simon