[systemd-devel] Thoughts on adapting daemons to use socket activation

[David Strauss]

I'm porting most of the daemons in use at Pantheon to support socket
activation. While Lennart's blog post on it is extremely helpful, I'd
like to share a concrete method that's been working for me with
complex daemons, including nginx and MySQL.

Here's a basic how-to based on what I've tried:

1. Hack the listen() code to just use the already-listening fd #3 that
systemd sends in for the most basic (single listener) socket
activation case. You can generally skip most of the listen() and
bind() stuff the code does to arrive at the file descriptor it will be
using in the rest of the code. Just set the fd uint to the number 3.
Make sure the configuration and <unit>.socket instantiate the same
type of socket (IPv4 TCP, IPv6 TCP, Unix domain). This isn't
shippable, but it should run. Until this is working reliably, there's
little point it doing a clean implementation. Expect some noise on
service/socket shutdown as the daemon tries to tear down the socket
represented by the fd that's actually managed by systemd.
2. Trace the path from the configuration file loading/CLI options all
the way to listen() calls that return a file descriptor. Usually, some
data structure representing how the daemon should listen is getting
populated and sent around. Extend it to include an integer file
descriptor which will be set to a non-NULL value if the "listener" is
already initialized and available as a file descriptor.
3. Update the configuration loading to allow listening on a file
descriptor, specified by number. If you want to be systemd-centric, if
can default to fd #3, since that's the 99% case for <unit>.socket
configurations.
4. Somewhere between configuration loading and socket initialization,
you may want to inspect the file descriptor to ensure that it's
present and a valid type. It may also be important to fill-in the
other parts of the "listener" data structure so the application knows
more about the socket behind the file descriptor. For example, nginx
likes to know what port and interface it's listening on. Done
properly, the post-socket-listening daemon should be (mostly)
oblivious to whether it inherited the socket from systemd or opened it
itself.
5. On shutdown, the daemon should skip the part of the socket
lifecycle where it stops listening and closes the fd/socket.

That all said, systemd could make this a bit easier with a few
features (future sprint?):

 * Better inspection functions for the file descriptors. Most daemons
assume you've configured a specific address, port, or Unix domain
socket path in their configuration files. Some, like web servers,
critically rely on that information when formulating responses to
clients or listing status information on their own configuration. It's
possible to get these out of the file descriptors, but it's kind of
boilerplate.
 * Functions that return the fully-formed sockaddr_un, sockaddr_in,
sockaddr_in6, and in6_addr data structures, fully populated, when
given only the fd number. A lot of daemons like having this structure
around for the reasons explained in the previous bullet.
 * A CLI utility for listening on a socket, launching a defined
executable (possibly only after a connection gets received), and
passing in the file descriptors in the same way a service+socket
would. This would make both manual and automated testing easier. Think
systemd-cat but for socket activation.

Additionally, socket activation could get rather interesting
capability if there were a middle-ground between single process per
connection and one process for all connections. Frameworks like
Twisted Python and node.js have built their own wrappers to do this in
various kludgy ways that involve a master process opening the main
socket and then passing file descriptors or other structures into the
fork()ed processes or using separate "load balancers" to spread the
requests out. This might be totally out of scope for systemd, though.

-- 
David Strauss
   | david at davidstrauss.net