DBus over the network; clustered machines acting as one; service discovery

[Avery Pennarun]

On Tue, Apr 8, 2008 at 10:04 PM, Shawn Rutledge
<shawn.t.rutledge at gmail.com> wrote:
>  Of course I know about some of them.  But you could also say that DBus
>  re-invented the wheel just a bit, no?  The Berlin project (later
>  renamed) was trying to use all-Corba all the time on the desktop, and
>  Gnome used to be heading that direction too.  For some reason it
>  stalled, and in general DBus is more popular for on-desktop IPC.  I
>  missed some of the details on why Gnome went that direction, etc.

DBus was created almost *explicitly* because Corba wasn't working out,
but not because it was "too much work" - they already had it working,
so replacing it was definitely more work.  It's worth looking into the
history of that specifically for educational purposes.

One of the main reasons Corba is a mess is that it tries to be
completely general and network-transparent about everything.  Because
of that problem, the DBus developers tend to err on the side of
keeping DBus very narrowly focussed on a few things, which I very much
respect.  It works, after all; I'd say DBus is probably much more
widely deployed by now than Corba is, since DBus is included with
virtually every Linux desktop and nowadays even some mobile devices.

I personally think DBus (the core protocol, anyway) is so well
designed that it will work great as a cross-computer networking layer.
 However, that does add a huge layer of complexity that people usually
don't expect, and that DBus has largely avoided by keeping its narrow
focus.

>   If I had been trying to solve the problem that DBus does, back before
>  it existed, I would probably have been inclined to start with sockets
>  (to get network transparency right at the outset) and later use Unix
>  sockets and/or shared memory as an optimization (following the same
>  kind of design evolution that X protocol did).

libdbus already uses Unix sockets, which are sufficiently abstracted
that it was easy to add TCP support on top.  Thus libdbus also
supports TCP sockets.  WvDBus is a reimplementation of the transport
layer that uses WvStreams, which supports many more kinds of sockets
transparently, including SSL.

Shared memory might be nice someday too for certain data-heavy
applications, although of course those aren't what DBus was designed
for either :)

>  So now I think after
>  having started on just one machine, eventually it will have to evolve
>  the other direction (going from one machine to clusters), and the end
>  result ought to be the same: it will do some or most of what Corba
>  did, and hopefully without the same cost.  It does try to be
>  efficient, right?

Corba tried to be efficient too.  When trying to solve the same
problems as Corba, one must have an idea of how they're going to avoid
the mess that Corba got into.  I gather that nobody has actually ever
succeeded at that, so be careful :)

My own goal in extending dbus to support SSL was much more modest: I
just wanted a decent transport layer for my network application
(versaplex), and I figured letting someone else design the extensible
message format was smarter than designing my own from scratch.  So
far, I think I was right.

Have fun,

Avery