D-Bus system services: systemd, Upstart, the future

[Lennart Poettering]

On Thu, 11.08.11 19:06, Simon McVittie (simon.mcvittie at collabora.co.uk) wrote:

Heya,

Sorry for not responding earlier, but you can scare me easily with long
mails like this ;-)

>   - some people have Upstart, but Upstart can run sysvinit scripts, so
>     the lowest-common-denominator for upstreams is to ship an init script
> 
>   - some people have runit/daemontools/..., they get to keep both pieces :-P
> 
>   - problem: systemd and Upstart both work better if you give them native
>     job formats, but we're giving them init scripts instead; this is
>     silly

Many upstream packages ship native systemd unit files these days, your
example Avahi for example does. And in Fedora 16 we have have converted
*all* init scripts to systemd unit files. My plan for Fedora 17 is to
have all bus services converted too.

> Current systemd/D-Bus integration (Fedora 15)
> =============================================
> 
> * merged upstream in D-Bus but apparently Lennart wants to change it

Not change, just add an additional, simpler mechanism. The additional,
simpler mechanism will be useful for systemd where compat with legacy
init systems is not necessary, since we basically get rid of the dbus
.service files. (i.e. Debian and suchlike who want to support multiple
init systems simultaneously will still have to ship a dbus .service file
for those systems).

> * System services which *don't* want to start on boot may ship a systemd
>   unit anyway, and just not symlink it
> 
> * Activatable system services require a dbus-1 system service definition,
>   which is a keyfile
> 
>   - to glue it to the systemd unit, it includes the name of the systemd
>     unit
> 
>   - problem: duplicate information in two files
> 
> * Service activation is done by systemd if the dbus-1 system service
>   definition specifies a systemd unit, or by dbus-daemon-launch-helper
>   otherwise
> 
>   - problem: services end up with different environment/management

Well, we cannot set a per-service environment in D-Bus, hence the
differences in the exec context are actually minimal. systemd cleans
things a bit better up before execing and getppid() will be 1 for
services spawned via systemd, but that should be it.

> Proposal 1: D-Bus service files are how you define D-Bus system services
> ========================================================================
> 
> We already have a search path:
> 
> * if you like starting things during early boot, you can already put them
>   in /lib/dbus-1/system-services (thanks to a merged patch from Lennart)
> 
> * Maybe borrow a concept from systemd by also searching
>   /etc/dbus-1/system-services, with highest precedence and some way
>   to say "include the corresponding file from /lib"?
> 
> * If you have separate /usr, the init system should tell dbus-daemon to
>   ReloadConfig after you mount it

I think we prepped the ground well enough now so that we can kill off this
idea entirely.

Splitting off /usr split is fine, as long as it is mounted in the
initrd. This is a consensus the big distributions have now agreed on,
and it is implemented this way at least on Fedora and SUSE. Splitting
off /usr and expecting it to be mounted after the initrd is not
supported anymore, and we should not think about it anymore.

Or in other words: if you want a split off /usr, then use an initrd
which mounts it for you before you jump into the real system. Split off
/usr without initrd is not supported. Due to this I am no longer pushing
my patches that allow D-Bus to be started without /usr around. 

> * AllowMethodCallsFromConsole? AllowMethodCallsFromGroups?

at_console needs to die.

> Proposal 2: dbus-daemon calls out to the init system
> ====================================================
> 
> In this proposal, dbus-daemon delegates ListActivatableNames to the
> init implementation and... that's about it.
> 
> Upstreams become sad, because they have to ship some combination of
> a traditional D-Bus service file, a systemd unit, an Upstart job and an
> init script, sufficient to cover every OS they want to target, possibly
> with different names. This is basically why I don't like this
> approach.

Well, the feature set of the init systems is very different. Trying to
use D-Bus service files as abstraction for service files because you
feel the need to support a gazillion of different init systems is a really
bad idea. The focus should be to design a sane system here, not one that
tries to make everybody happy where it is clear that not everybody can
be made happy.

It is my explicit intention to unify all kinds of services we have under
the umbrella of systemd. If it traditionally was a sysv service, an
inetd service, a bus service or a cron job (of a certain kind) shouldn't
matter anymore, it should now just be a systemd service configured with
the same simple configuration language with the same powerful options,
and the same supervisor, regardless under which category it was
previously sorted. Whether it uses the bus or uses sockets is just an
implementation detail. It is also an implementation detail whether a
service is activated at boot, by sockets, by the bus, by hotplug, by
file creation, by timers or multiple of these triggers at the same
time. In fact, many daemons will use multiple communication interfaces,
and have a number of triggers (Avahi for example uses the bus, and a
local AF_UNIX socket (for the NSS stuff) and is pulled in by hw, by
socket and by bus activation).

So from my perspective Proposal 2 is the only way to go. Proposal 1 is
not at all interesting for us. It would place D-Bus in the focus and
pretend there was something specific about a "D-Bus service", which
however is not the case. There are just "services", some of which happen
to speak D-Bus, among various other communication technologies.

I can understand that you care for Debian, and its interest in supporting
every init system on this planet. But I think it's Debian's job to do
the work, and should not cause us to fuck up the design of D-Bus and the
OS.

Or to put this in even other words: the time of supporting all kinds of
compatibilty syntaxes for services is over for us. systemd is not an
abstraction layer for various service definition files. Right now, in
systemd we only support one legacy format: SysV init scripts, because it
is so deeply entrenched that we cannot really go without it. However
even for that we are slowly getting ready to deemphesize it a bit. For
example, soonishly we'll drop support for init scripts for the "S" or
"boot" runlevels, i.e. "early boot". We will then allow SysV services
only for runlevels 2-5. Next we'll rip out SysV support from PID 1 and
move it into a generator binary which is called at boot and simply
generates dynamic throw-away services that invoke the old sysv
scripts. All that in preparation to get rid of SysV entirely in a couple
of years. That path is clear, and it would be diametrically against our
intentions here to add in another 1st class unit file syntax. We want
fewer service file syntaxes, not more.

It also has something to do with simplicity: in a clean system we better
avoid having two kinds of files with the suffix ".service". Since we
can't get rid of the systemd .service files it is my hope to get rid of
the D-Bus .service files, instead.

> Comments? Alternative proposals? Half-bricks? Cakes?

Quite frankly, I am not fond at all of the idea of emphasizing D-Bus
.service files. It is my explicit intention to deemphasize them.

And implementing your idea #1 is much more complex than it might
appear. For example, in systemd we strictly don't do automatic config
reloading, which D-Bus does. It's very hard and inefficient to support
the current behaviour of .service files within systemd that way.

I am open to abstracting the activation mechanism a bit so that the
"systemd" name is removed, if that's what the Upstart/Debian folks want
(I am even happy to do this without keeping compat here). However, I am
unwilling to agree to make D-Bus .service files the new gold standard.

Sorry if this is disappointing,

Lennart

-- 
Lennart Poettering - Red Hat, Inc.