D-Bus system services: systemd, Upstart, the future

[Simon McVittie]

I said in the "D-Bus mini-summit" thread that I thought systemd (and friends)
service activation deserved its own thread. Here's the beginning of that
thread.

If Lennart goes ahead with plans to manage session services with an
unprivileged systemd, much of this potentially applies to the session bus too,
but I'm concentrating on the system bus for now.

Hopefully my descriptions of systemd and Upstart behaviour are broadly
correct, but I'm probably wrong about some details; hopefully nothing
significant.

Traditional situation ("just D-Bus and sysvinit"; Debian 6, Fedora 14)
======================================================================

* Non-trivial system services require changes to the default-deny security
  policy. These are implemented via files in /etc/dbus-1/system.d.

  - the upstream author of the service knows what default policy should
    normally be applied, and supplies a typical policy

  - distributions might change it:

    - trivial changes, e.g. does Avahi run as username avahi, avahi-daemon,
      _avahi, Debian-avahi, daemon (not encouraged), or nobody (bad idea)?

    - overall distro policy: err on the side of "users can get things done
      but can DoS each other" or "users can't get things, including DoS, done"?
      (of course, everyone should aim for "useful but DoS-resistant" but
      that's not always feasible)

    - in Debian 5 era it was typical for each upstream policy
      "allow at_console users to do foo" to be supplemented with a policy
      "allow users in group foodev to do foo" - admittedly, this was a
      workaround for slow uptake of ConsoleKit etc.

  - system administrators can also change it for local policy

  - current/future trend: service authors are encouraged to allow most/all
    method calls at dbus-daemon level, then perform discretionary access
    control in their service by calling out to PolicyKit, which can be much
    finer-grained/cleverer than dbus-daemon policies

  - problem: the policy language is such that innocuous-looking service
    files have unforeseen effects (cf. CVE-2008-0595, CVE-2008-4311)

  - problem: the policy language is far too powerful - any service can
    cause unexpected behavioural changes for any other service
    (in Maemo we've had problems with a service enabling system-bus
    eavesdropping globally, causing problems similar to fd.o #37890, because
    its developers set that up for initial work and never got round to
    fixing it)

* System services which have desirable non-D-Bus side-effects (e.g. Avahi
  advertising mylaptop.local, NetworkManager connecting to wired networks
  before you log in) require an init script, which is a boilerplate-filled
  shell script which might even work sometimes

  - it has a short name similar to the RPM/dpkg package and/or executable,
    like "avahi"

  - some people have systemd, but systemd can run sysvinit scripts, so
    the lowest-common-denominator for upstreams is to ship an init script

  - some people have Upstart, but Upstart can run sysvinit scripts, so
    the lowest-common-denominator for upstreams is to ship an init script

  - some people have runit/daemontools/..., they get to keep both pieces :-P

  - problem: systemd and Upstart both work better if you give them native
    job formats, but we're giving them init scripts instead; this is silly

* Activatable system services (e.g. o.fd.Accounts) require a dbus-1
  system service definition, which is a keyfile (generalization of .desktop,
  parsed with GKeyFile or equivalent)

  - always named after the namespaced D-Bus service name - uniqueness
    guaranteed

  - problem: this basically duplicates the init script (if provided)

* Service activation is done by dbus-daemon-launch-helper, based on the
  dbus-1 system service definition

  - you can set User

  - ... and basically nothing else

  - UpdateActivationEnvironment is disallowed by default (even for root),
    and is basically a kludge anyway - let's not support it on the system bus

Current systemd/D-Bus integration (Fedora 15)
=============================================

* merged upstream in D-Bus but apparently Lennart wants to change it

* The security policy is the same as above, with the same problems

* System services which want to start on boot ship an init script, and/or
  a systemd unit of the same name which is configured to auto-start

  - if the systemd unit exists, the init script is ignored - backwards
    compatibility with sysvinit + better hookup with systemd,
    everyone wins!

* System services which *don't* want to start on boot may ship a systemd
  unit anyway, and just not symlink it

* Activatable system services require a dbus-1 system service definition,
  which is a keyfile

  - to glue it to the systemd unit, it includes the name of the systemd
    unit

  - problem: duplicate information in two files

* Service activation is done by systemd if the dbus-1 system service
  definition specifies a systemd unit, or by dbus-daemon-launch-helper
  otherwise

  - problem: services end up with different environment/management

Current Upstart/D-Bus integration (Ubuntu 11.x)
===============================================

* not merged, fd.o #34526

  - problem: Ubuntu carrying around a significant diff forever doesn't
    benefit anyone...

  - but conversely, having three or more distinct code paths in dbus
    (dbus-activation-helper, systemd, Upstart), only one of which is
    unit-tested, doesn't help anyone either, least of all
    D-Bus' maintainers

* System services which want to start on boot ship an Upstart job,
  and refrain from installing an init script because Upstart would
  try to run that too - it doesn't know which init script is superseded
  by which Upstart job

  - (Scott, is this still true?)

  - a design difference from systemd is that the names of Upstart jobs
    are not considered to be syntactically significant, and in particular
    are not assumed to match what the init script used to be called

  - the job declares itself to start on boot, and also to
    "start on dbus-activation com.example.MyService"

  - there's nothing to stop you starting multiple things triggered by
    one dbus-activation message (although I have no idea why you'd want to)

    - this is normally a feature, but for D-Bus' usage it may be closer to
      a bug - if you have n competing implementations of a common interface,
      presumably the first one to take its bus name wins?

* System services which *don't* want to start on boot may ship an Upstart
  job which doesn't start on boot, but does "start on dbus-activation"

* Service activation is done exclusively by Upstart

  - in the Upstart patch-set, dbus-daemon tells Upstart the User
    and Exec from the dbus-1 system service definition, so Upstart
    can/will use those if it doesn't know any better way to do it

Observations
============

* Upstart (or something shipped with it) contains specific code to interact
  with dbus

  - start on dbus-activation "com.example.badger"

  - listening for messages

* systemd (or something shipped with it) contains specific code to interact
  with dbus

  - org.freedesktop.systemd1

  - listening for messages

* dbus-daemon's search path for service files is essentially hard-coded
  (i.e. fixed, at least for each distribution), because it vaguely tries to
  respect XDG_DATA_DIRS, but that doesn't actually work anyway (fd.o #21620)
  and it's not at all clear to me (or Lennart) that it *should* work

Proposal 1: D-Bus service files are how you define D-Bus system services
========================================================================

In this proposal, D-Bus service files are how you have always defined
D-Bus system services, so they should continue to be how you define D-Bus
services, and init authors who want to implement something better than
dbus-daemon-launch-helper will just have to live with it.

Upstreams have to provide them anyway, if they want to be compatible
with "plain ordinary D-Bus", so this is as easy for upstreams as anything
else would be.

The down side is that upstreams who want their service started on boot
in an init-agnostic way would have to provide an init script to do it
(but this would be unnecessary for systemd and Upstart, assuming they
can read a "start me on boot" hint from the D-Bus service file).
The init script could just use dbus-send or something to get the
service started, though? (Avahi doesn't do this, at least in Debian - but
it could.)

systemd, Upstart and anyone else who wants to join in can join D-Bus
and listen for requests; but they have to have D-Bus-specific code already,
so that's not a regression. dbus-daemon will expect this if run with
a new option --external-activation=com.example.MyInit. It's easy for init
systems to arrange to pass the right command line arguments to dbus-daemon,
because the init system starts the dbus-daemon anyway - D-Bus would have
"dbus-daemon --external-activation=org.freedesktop.systemd1" in its systemd
unit, and something like "dbus-daemon --external-activation=com.ubuntu.Upstart"
in its Upstart job.

Compatibility: --systemd-activation would become an alias
for --external-activation=org.freedesktop.systemd1.

We already have a search path:

* if you like starting things during early boot, you can already put them
  in /lib/dbus-1/system-services (thanks to a merged patch from Lennart)

* Maybe borrow a concept from systemd by also searching
  /etc/dbus-1/system-services, with highest precedence and some way
  to say "include the corresponding file from /lib"?

* If you have separate /usr, the init system should tell dbus-daemon to
  ReloadConfig after you mount it

  - or make /usr/share/dbus-1/system-services a symlink to
    /lib/dbus-1/system-services, make /usr/local/share/dbus-1/system-services
    a symlink to /etc/dbus-1/system-services, and move all your service files

systemd, Upstart and anyone else who wants to join in is expected to search
the same directories that D-Bus does - if we wanted to, we could make this
tautologically true, by having them pass
something like --prefixes=/etc:/lib:/usr/local/share:/usr/share to the
dbus-daemon.

systemd would presumably change from "use me for all services that have
opted in" to "use me for all services".

Because the init system is directly reading system services' D-Bus .service
files, it can use its own namespaced group in the keyfile to get whatever
extra information it wants to. For init systems that use keyfiles anyway
(hi, Lennart!) it can even be in the init system's usual syntax;
init systems that use their own format (Upstart) can either invent a
keyfile-compatible syntax, or have a second file.

Any extra information in the D-Bus .service file should be entirely optional:
the service must work without it, but might not work optimally (for instance,
if the service-activator doesn't understand the systemd "please adjust my
OOM-kill score" setting, then the OOM-kill score wouldn't be adjusted).

systemd units being started on boot could be implemented by symlinking the
D-Bus service file into a systemd-controlled location, if it's simultaneously
a systemd unit and a D-Bus service file anyway. This leaves only Upstart
needing to know, separately, how to get things started on boot?

dbus-daemon-launch-helper could go in several directions:

* turn into a special D-Bus service that runs as root from an init script
  (no need for an Upstart job or a systemd unit because it's unwanted in
  both of those situations!), listens for messages from the dbus-daemon
  (only!), and performs activation just like systemd does now (but in a
  substantially simpler way)

  - this means running quite a lot of libdbus as root, which increases
    security surface-area, but systemd and Upstart, and other daemons
    like NetworkManager, already do that anyway

  - "only root may take that name" can be enforced by the same security policy
    that e.g. NetworkManager already needs; during regression tests we'd omit
    that part of the policy, and the exact same executable would be used, but
    unprivileged!

  - I personally think this is the best option, it means literally everyone
    is using what's now the systemd code path - better test coverage
    by basically turning dbus-daemon-launch-helper into a toy init
    implementation, which is effectively what it is anyway

* turn into a special D-Bus service that is forked/exec'd by the dbus-daemon
  (4754 root:messagebus, like it is now)

  - more security surface area, like above

  - setuid, like below

  - minor second code path to do the startup
  
  - I think this is the worst of both worlds

* remain as a second code path inside dbus-daemon (which does a fork/exec
  for each activation), and just do state sanitization / exec the service

  - still needs to be setuid - always scary

  - needs a different build of itself for unit tests - I never liked this

  - second code path :-(

Proposal 1a: put ACL policy in the .service file
================================================

As for proposal 1, but also let the .service file control the policy,
deprecating (that usage of) system.d. This requires that the local sysadmin
can override it with files in /etc.

It could be very simple:

* as a matter of general system-bus policy, broadcasts are public - de facto
  truth, let's just make that something that's Officially Always True

* the user named in User=foo is allowed to own the name

* maybe add AlsoAllowUser=root for debugging (or not - root
  can use sudo or something, or just always be allowed)

* AllowAllMethodCalls=true (99% of services want this - either they're open
  to all anyway, or they use PolicyKit)

* AllowMethodCallsFromConsole? AllowMethodCallsFromGroups?

* in practice nobody uses policies like "only if you're either in the audio
  group, or at the console on a Tuesday afternoon", and if they do, they
  should be using PolicyKit

* the legacy policy language could remain for backwards compat and
  complex policies; actions would be allowed if either the service file
  says you can, or the legacy policy language says you can (?)

  - or the policy resulting from the .service file could be inserted
    into the search order after "99.conf" but before "a.conf", or something?

Proposal 2: dbus-daemon calls out to the init system
====================================================

In this proposal, dbus-daemon delegates ListActivatableNames to the
init implementation and... that's about it.

Upstreams become sad, because they have to ship some combination of
a traditional D-Bus service file, a systemd unit, an Upstart job and an
init script, sufficient to cover every OS they want to target, possibly
with different names. This is basically why I don't like this approach.

Again, the dbus-daemon-launch-helper case could either be done by
turning it into a toy init implementation, or by keeping a second code path
for it.

systemd would presumably still change from "use me for all services that have
opted in" to "use me for all services".

-------------

Comments? Alternative proposals? Half-bricks? Cakes?

Regards,
    S