Service discovery on message bus

[Simon McVittie]

On Fri, 23 Mar 2018 at 13:47:07 +0100, Umut Tezduyar Lindskog wrote:
> I am wondering if there has been any discussion around making
> interface discovery part of the message bus protocol?

Well-known name discovery provides either service discovery
or interface/protocol discovery, depending on how you look at
it. When defining a well-known name like com.example.Calculator or
org.freedesktop.FileManager, it's very common to define a protocol (an
interface in the general, broad sense of the word, but not the D-Bus
jargon meaning of interface) that is expected to be implemented by every
provider of that well-known name.

That protocol will typically take a form similar to "there is a
well-known 'entry point' object exported at /foo, which implements
the necessary interface com.example.Bar to let you discover the rest
of the service's functionality".

For a simple real-world example, consider the systemd Manager.
When you see org.freedesktop.systemd1 on the bus, you know that it has
an object at /org/freedesktop/systemd1 that implements
org.freedesktop.systemd1.Manager, because systemd's own documentation
says so.

For a more complex real-world example, see Telepathy or MPRIS2, both
of which define a protocol in which a family of well-known bus names
can be relied on to implement particular interfaces at particular
object paths. Recent (post-2012) versions of dbus support match
rule qualifiers and system-bus policy rules for entire namespaces of
well-known bus names, which is very suitable for the Telepathy/MPRIS2
pattern.

> Message bus cannot answer the question of which service implement what
> interface out of the box.

No, it can't. However, if you have the necessary application-specific
knowledge to make practical use of a protocol, then you should also have
the necessary application-specific knowledge to find an implementation
of that protocol (and if you don't, that's a bug in the documentation
you've read, which its authors should correct).

The only valid use-case that I'm aware of for manipulating objects and
interfaces without understanding what they mean is implementing a generic
debugging tool like d-feet, which can make use of Introspect() to discover
objects and interfaces. (We do not recommend using Introspect() in
production code: it's there for debuggers.)

> Services have to do their own discovery of services
> (ListNames/ListActivatableNames) followed up with introspection of
> each objects in the service or relaying on the fact that services are
> implementing org.freedesktop.DBus.ObjectManager interface. This
> approach is very resource consuming.

If services had to notify the dbus-daemon about every object that they
created or destroyed, and had to notify the dbus-daemon about all the
interfaces that those objects have, I don't think that would consume
fewer resources: the dbus-daemon would have to maintain a large data
structure for all the objects and interfaces (even if nobody is currently
observing that data structure!), and querying that data structure would
involve messages remarkably similar to the messages that are currently
used to inspect individual services' object trees.

For example, for the systemd Manager, there's one object per unit and
one transient object per job. Mirroring that hierarchy from systemd's
memory space into the dbus-daemon's memory space doesn't seem
productive: a client would have to make the same number of IPC calls
to inspect the object hierarchy either way, maintaining the cache
would cause additional bus traffic, and there would be one copy of the
object hierarchy in systemd and another in the dbus-daemon even if
nobody is currently communicating with systemd via D-Bus.

For something like MPRIS2, it's worse: each of probably thousands of
songs in the user's music collection is (conceptually) represented by an
object path (the object has no defined methods, signals or properties,
but it exists). Mirroring all of *those* in the dbus-daemon's memory
space does not seem like an efficient use of resources at all.

> A singelton service can do something similar to generate a registry
> but then why would this service live in a separate connection instead
> of org.freedesktop.DBus?

Separation of responsibilities: for anything provided by the
dbus-daemon, you'll pay the performance and complexity cost regardless
of whether anyone else is using it. The dbus-daemon implements just
enough functionality to do message routing and service discovery, and
leaves the rest to the peers on the bus.

> BTW, I believe discovering services has a race. User asks for
> org.freedesktop.DBus.ListNames but service A is not in this list
> because it is not activated. Then A gets activated and user asks for
> org.freedesktop.DBus.ListActivatableNames to non activated services.
> Obviously A is not in this list.

ListActivatableNames returns all names that could be activated, regardless
of whether they would have appeared in ListNames or not. There will
normally be a significant overlap between the two lists.

Even if ListActivatableNames and ListNames returned disjoint results,
you could equally well subscribe to NameOwnerChanged signals for the name
or names of interest before calling ListNames, which would tell you when
A was activated. If you find yourself calling ListNames for anything
other than a "one-shot" diagnostic tool, then you almost certainly should
be subscribing to a subset of NameOwnerChanged signals anyway, to track
changes to the set of visible names. See telepathy-mission-control for
an example of doing this.

There *is* a design flaw around ListActivatableNames, which is that it
has no change-notification: when the dbus-daemon reloads configuration,
it doesn't provide any indication that the set of activatable names
has changed. This has not generally been a major concern; it would be
a relatively simple project to add a change-notification signal, but
in dbus' ~ 15 year history, nobody has yet been sufficiently troubled by
its absence to do so.

Finally, the recommended pattern for discovering services that your
application wishes to use does not actually involve ListNames. To avoid
time-of-check/time-of-use errors, if your application wishes to make
use of a service, the recommended way to proceed is to try to
communicate with that service by its well-known name (the "leap before
you look" or "better to ask forgiveness than permission" model). If
the service is available, it will reply; if not, you'll get an error
message from the dbus-daemon, and the error name in that message
can be used to find out why the service is unavailable.

    smcv