state of dbus and guadec dbus meeting notes

[Havoc Pennington]

Hi,

A bunch of stuff I remember from our discussions here, I took a few 
notes but they were incomplete and I don't have them with me right now 
anyhow.

List of topics; if you want to discuss one, probably you should start a 
thread per topic and change the subject line.

Also, add stuff to the list that I forgot.

I'll just put in my editorial comments, since, well why not. As always 
they are subject to change.

In the category of "decisions":

- we are likely to just switch to MIT/X11 though this hinges on someone
   mailing all the AUTHORS. vs. the AFL which is X11-style anyhow, this
   just drops the patent clause. While I am not convinced GPL/AFL causes
   any problems not caused by LGPL (see my recent blog post), it seems
   we can just avoid this argument forever by going with MIT/X11.

- we are going to move the spec xml file to its own module, with a
   simple Makefile to build the HTML. update-dbus-docs.sh will be
   fixed to also check out the spec and publish it. The git
   module for the spec will be in the dbus group on freedesktop.org so
   have the same committers. (someone please fix the commits list...)

In the category of "ideas we talked about":

- a feature negotiation thing OR versioning thing in the auth protocol
   and in the libdbus API, needs adding before most protocol changes are
   possible

   - if someone works on this, I would suggest trying to avoid
     introducing another round trip in the auth protocol for the
     "normal case"

- an "index" into arrays on the wire, giving O(1) array indexing. We
   discussed doing this as a header field "array offset table" (requires
   no feature negotiation / protocol change) or as an actual protocol
   change negotiated at auth time. This would allow e.g. the "dconf"
   config system Ryan is working on to avoid ever copying the DBusMessage
   or equivalent.

   I find it mildly cleaner to do this as an optional optimization in the
   message header, but I suppose it's a matter of taste and I don't have
   especially strong feelings. It would also be easier to code in libdbus
   as a header field.

- Type system extensions, many were proposed. All require the feature
   negotiation in the auth protocol as a first step.

    - add single-precision float
      - makes type system more completely map to C# and Java, though I
        argue the dbus type system is intended to be an intersection not
        union of systems
      - the only cited use-case is GL or other 3D graphics, nobody uses
        float otherwise.
      - I don't think there's an especially strong argument either way;
        I probably wouldn't break the protocol just to add this, but
        if we're doing one of the below things anyway perhaps.
      - I know there are archive threads on this which someone should dig
        out in case we forgot good reasons for omitting float.

    - add "maybe" types, essentially typed NULL, i.e. "ms" = "maybe
      string" = "string or null"
      - C/Java/C#/C++ binding for "maybe int" and similar is ugly
      - there are a fair number of hackarounds out there for this,
        and it's a simple feature, so I like it
      - perhaps it should only be for types that are objects/nullable
        in most common languages (the string-ish types and container
        types)?

    - add discriminated unions, the signature would contain a list of
      single complete types and the marshaled value includes the index
      of the actual type, then the value.
      - this is enough the same as just having an int then a variant, I'm
        not sure it's worth it, but it does save some manual
        type-checking in application code
      - maybe it hinges on how much of a pain it is to implement

    - add enum names to the introspection format (not wire format)
      - allows bindings to more nicely interpret an integer

    - we talked about empty structs, I think the old thread on this
      should be dug up, it may have stuff we forgot. generally
      I got the sense nobody wants this if we have the "maybe"
      types (i.e. null)

- some discussion of use of XML not binary dbus types for introspection
   - xml more extensible
   - xml adds another thing to know to parse
   - my opinion is that changing this adds more confusion than it's
     worth, it could have been different but works fine

- in any case there are a bunch of introspection XML extensions floating
   around that probably need standardizing

- I argued that the primary current pain for app developers is not any
   of the type system tweaks and so forth, but the need for convenience
   API around for example:
   - single-instance apps
   - tracking another bus name
   - tracking "clients using my service"

- we debated whether to abort on bad use of the API or add DBusError
   throughout the API, for libdbus at least I think this is fairly
   academic since it would be a giant pain to go through duplicating the
   API with DBusError versions

- the bus daemon needs to be optimized to support more match rules
   without O(n) performance degradation, then the match rule limit should
   be raised

- we discussed removing the requirement that interface names contain a
   "." so e.g. a Java interface in the default package could be mapped
   without special case. I don't have a strong opinion here I guess.

- there's a byte-swap bug we may have forgotten to fix with a big-endian
   daemon and little-endian client, we need to find this in archives and
   figure it out

- lots of desire to write custom dbus client implementations, since
   libdbus has a lot of extra complexity for the daemon that clients
   don't need, and e.g. aborting on bad API use is right for C but
   a pain for bindings which would prefer to have a DBusError passed
   to every entry point.

   - one school of thought, which I tend to agree with, is that a binding
     can fairly easily have its own message queue and do its own
     marshaling but that evolving dbus will be a lot more painful if
     everyone reimplements address parsing/listening/connection, locating
     the bus, and the auth protocol. We expect to add more address
     formats such as Dan Berrange's outstanding patch for TCP, and we
     just added e.g. ANONYMOUS auth.

     So the approach in this scenario would be to have a common C lib
     that had a blocking call to open and auth a connection, then
     returned a descriptor.

     The hard part is designing this API and refactoring libdbus to use
     it, the bikeshed question is whether to split libdbus into this
     stuff and the DBusMessage/dbus-bus.h stuff. Splitting libdbus is
     pretty problematic due to DBusString, dbus-sysdeps.c etc. and
     also uses more resources due to another shared lib with associated
     per-process descriptor and what have you. Anyway, can figure this
     out.

   - for C# and Java, a pure no-C implementation makes a lot of sense to
     me since C has high cost. For languages that have a bunch of
     bindings to C libs anyway, the extra cost to the overall desktop
     world in maintaining the auth/connection-opening stuff many times
     over is weird to me. But people are of course welcome to spend
     their time as they see fit ;-)

   - as a general observation, I think the enthusiasm for
     reimplementation is somewhat at odds with the enthusiasm for
     adding lots of new features (the features are progressively
     harder to add and start using as we have more implementations,
     especially if one or two widely-used implementations start to become
     unmaintained which is inevitable sooner or later).

   - on the other hand I think slowing feature evolution to a near stop
     at some point is probably a good thing

   - more implementations do increase our exposure to bugs and security
     bugs, due to more code and less-good maintainer:code ratio.

- generally a lot of the transport / bus-location stuff is undefined on
   Windows, which causes problems for the Mono reimplementation as well.
   Someone needs to specify sane mechanisms for e.g. starting up and
   locating the bus daemon on Windows and put them in the spec.

- we want prefix or glob matching on argN in signal match rules, for
   dconf. not really sure of the details yet.

- we talked some about the recent thread about two-stage name releasing
   (the fact that a name might be both for routing and for a lock on
    some backend resource).
    - my feeling is that "use two different names" is a sane/fine
      solution, but another solution might be interesting if it's simple

- the largest gap in the spec is a detailed discussion of dbus-daemon
   semantics

- I know I'm forgetting some stuff, follow up and add it


Outside of the official DBus meetings I talked to Will Walker from
GNOME a11y, and he felt it was a ton of work without real purpose to
drop ORBit; I tend to agree, but we discussed dropping
bonobo-activation in favor of more reliable dbus-based activation
that gets session scoping correct.


We didn't discuss the 1.2 release, but for general information I was 
thinking it would have:

- system activation, which I reviewed with Richard yesterday and he is
   going to commit shortly

- the patch from Dan Berrange to fix up the TCP addressing, which needed
   one more iteration iirc. Dan if you don't have time let me know and I
   will try to get to it.

That's pretty much it, I suggest we bake those things and release.


On the GLib binding front, there may be a new approach/angle that takes 
into account Ryan's DValue stuff and cute varargs hacks, GObject 
introspection and Rob's C parser stuff, and/or the convenience API stuff 
I was working on (mentioned above). I don't know exactly how this all 
evolves.

For GNOME at least I would say fixing the GLib bindings in some or all 
of these ways is _the_ critical dbus issue, more important by far than 
all the protocol tweaks and so forth. Right now it's too hard for apps 
to use (and they tend to use it incorrectly or just use the raw libdbus 
API as a consequence).

I would say that the focus of bindings to date has been on marshaling, 
but perhaps especially with cute varargs hacks and DValue, that almost 
becomes the _least_ important thing to do.


Outside of GNOME, the Windows port (in particular, the specification of 
bus location etc.) is one of the higher-priority items in my mind.


Just to emphasize some of the cautions earlier in this mail, while 
there's a long list of tweaks here, most of them aren't as important as 
other things we could be doing in free software, and a 
flaky/ever-changing IPC system is truly a nightmare for making a desktop 
work. KDE did build a whole working desktop by heavily using DCOP which 
was much simpler than dbus.

It would not hurt us to take all this sort of slowly and conservatively, 
and insist on things like test suite coverage as we write dbus-related 
modules, whether they are in the reference implementation codebase or not.

I'd suggest focusing most on work that will dramatically reduce number 
of lines of code in apps and/or app correctness.

That said there are lots of good ideas and worthwhile things to do here 
and it's great to see so much interest (I had no idea prior to GUADEC 
that people were paying lots of attention to dbus, to be honest).

Havoc