Starting the kdbus discussions

[Lennart Poettering]

On Sun, 29.12.13 20:42, Ted Gould (ted at gould.cx) wrote:

> > Nope. We provide quite complete compatibility, there are only very few
> > exceptions visible to app programmers. One is the different policy language
> > use for the system bus, but even for that we provide full compatbility
> > if people connect via the good old AF_UNIX protocol.
> 
> From my understanding, every application that ships a service file today
> will have to change that to be a systemd service if they expect to use
> kdbus directly.  Not sure if the kdbus to dbus proxy would support the
> dbus service files.  Seems that application expecting to use both would
> need to ship both files and ensure compatibility there.

Well, it's not that different from today. On Fedora at least we ship a
systemd service file each for all bus-activated system service these days,
and the dbus activation files are mostly just compatibility. So they
already are systemd unit files, and they just continue to be, its just a
bit more mandatory.

> > The flags are no "arbitrarily" inverted. We simply dropped the weird
> > inversion that dbus1 had there in the first place. Moreover, this is
> > not visible to the outside anyway, as applications should connect via a
> > library such as libsystemd-bus, or gdbus to the thing, and those library
> > trivially abstract the app developer away from this diffrence.
> 
> Sure, if you were making a new protocol that makes sense.

I think you are misunderstanding something here: the payload header
actually follows the exact same definition as dbus1 does for this field,
including the weirdly inverted bits. However, the kernel gets via its
ioctl() a slightly different set of control bits independently of this,
which indicate similar information. Since those bits however don't
translate 1:1 anyway to those payload fields, we decided to clean them
up and made them non-inverted.

So again: the fixed dbus header sent via kdbus as transport is actually
bit-for-bit identical to classic dbus1. It's only the thin ioctl()
metadata we tell the kernel that has cleaned this bits up.

See this:

http://cgit.freedesktop.org/systemd/systemd/plain/src/libsystemd-bus/GVARIANT-SERIALIZATION

As you can see: the first 16 bytes are 100% identical between dbus1 and
gvariant messages. Only beyond that the gvariant marshalling results in
different serialization.

> > The serialization format is mostly not visible to the outside anyway in
> > gdbus/libdbus. All the signature logic of GVariant and dbus1 marshalling
> > is pretty much identical however. In fact, gdbus converts everything
> > from/to GVariant anyway in userspace already, even on dbus1
> > marshalling. Morever, we even provide compatibility with dbus1
> > marshalling, again in the bus proxy. If you complain that gvariant was
> > so different from dbus1, then you probably shouldn't use glib's gdbus
> > either.
> 
> Not everyone who connects to DBus uses GDBus.  Optimizing the service
> for a particular client library seems odd.  And it seems odd to say that
> "this is DBus" with a different serialization format.

There are nowadays pretty much exactly three bus libraries in use:
libdbus1, gdbus and libsystemd-bus (the latter currently only used by
systemd). The other libraries are either wrappers around libdbus1 or not
really maintained anymore, enough that I am happy to ignore them here.

Ryan is doing the porting work for gdbus, and the Samsung guys are doing
the porting work for libdbus1. With that in place, we have support from
all three libs.

> > systemd repo and use systemd apis and technology, that is true. We also
> > provide an alternative low-level D-Bus client library in systemd
> > (libsystemd-bus) which can connect to both classic dbus1 and kdbus
> > directly, and abstracts all differences away.
> 
> I realize that you put them there for easier development, but do you
> expect things like the compatibility proxy to remain in the systemd
> repository?  Seems like it should release with dbus-daemon to maximize
> compatibility.

Nope, they are in systemd because that's what makes sense. It's systemd
in PID 1 that issues the ioctl to create the system bus. It's systemd
which will upload the dbus policy into the kernel at the same time as
uploading the selinux, ima or smack policy. systemd learnt a new unit
type called ".busname" which is how we expose bus activation as high
level objects in systemd with uniform behaviour, similar to how socket
activation is already covered. The bus proxy is a socket activated
systemd service, the bus driver a bus activated systemd service. We want
IPC to be an integral, uniformly integrated part of the OS, that's why
it is part of systemd. (And anyway, I am also not interested in other
systems, so why would I care?)

I can certainly understand that you feel left out on Ubuntu with
this. But please understand that that's hardly my problem, and we do
stay compatible pretty much completely, to make life easy for you that
you can stick with dbus-daemon.

The alternative of course is to develop a different kdbus userspace for
Ubuntu, and to make this easy for you I am posting this so that we can
get this documented in the spec, and we all can do the same on the
backend.

> > Note in particular that when sockets are used as transport the
> > unmodified dbus1 marshalling is used. This is extensively used by
> > systemd to implement things like "systemctl -H" or "systemctl -M" which
> > allows connections to another host via ssh, or connections to a local
> > container via namespace-traversing sockets.
> 
> The behaviors will be different.  Timings will be different.  The bugs
> will be different.  Implying that all of this can be abstracted away is
> unrealistic.

Well, the spec exists precisely to minimize these differences and to
allow multiple implementations while staying compatible. I am trying to
work towards that goal. Are you?

> > > I realize this is a work in progress, but it seems to me that it's not a
> > > feasible replacement for DBus and in fact something different that is
> > > intended to work in a more limited set of systems.
> > 
> > Yeah, it might not be an appropriate choice for you/Ubuntu, but I see no
> > reason why this would even matter to you, given that everybody else can
> > use this just fine and things are pretty much compatible for app
> > developers.
> > 
> > Ubuntu is welcome to stay with classic dbus-daemon/dbus1, the APIs
> > exposed by gdbus and so on abstract the backend away. 
> 
> Certainly.  But there is also a discussion of "what is DBus?" which the
> DBus community needs to decide.  For me, this seems more like
> systemd-bus than DBus.

I fail to see what you are trying to achieve here?

I mean, we either can get this discussed and done and into the spec and
we agree on common ground and open interfaces, or we can part ways, not
care about dbus1, do our thing independently of the old spec, not
include you in any discussions and define a new spec, and make all our
userspace work with only that. This would certainly cement the
seperation between Ubuntu and friends and the rest of the world even
further and make it even harder for you to use our technologies. I
cannot see how that would be in your interest...

Either you partake in the discussions and influence the spec at least
somewhat, or you tell us to go away and don't influence anything. I
somehow have the suspicion here though that kdbus is more interesting to
most people than classic dbus1 pretty soon, so excuse my arrogance here,
but good luck with that.

Lennart

-- 
Lennart Poettering, Red Hat