[systemd-devel] Parallel startup with sockets and without killing the machine?

Mon May 9 12:16:25 PDT 2011

Hi Scott,

Some answers below, but consider reading Lennart's reply first. I'll try to
cover some points he did not in his reply.

On Mon, May 9, 2011 at 1:10 PM, Scott James Remnant <scott at netsplit.com>wrote:

> Hey, I've been reading through the documentation on systemd and the Mailing
> List Archives and have had some questions, so I figured I'd post them here.
> These might be fairly complicated or unusual cases, and there may be no good
> answers now, and maybe no good answers ever, but I'd still like to get an
> idea of what you think.
>
> First up, parallel startup using sockets.
>
> We know that the theory of this goes something like this: you have a
> service that many other services and processes depend on, and connect to via
> a socket. Rather than manually express that dependency in the configuration,
> you instead have the socket created by the init daemon in advance. Now you
> simply start the service and all of the other services and processes
> together, and let the blocking nature of those sockets (or the software
> polling a non-blocking socket) sort out the ordering for you.
>
> This seems to assume that it's a free cost to start a service or process
> and have it block on a socket. But that isn't true, to get to the point
> where it connects to the socket, that service or process first has to be
> loaded from disk, its shared libraries mapped and relocated, it probably has
> to do a bunch of initialization including reading its own configuration
> files, etc.
>

Yes and that's exactly the reason it's good to start them before the other
service is fully ready to be used. Say that while DBus is starting due
UDisks request, UDisks may be parsing useful information from system (udev
tables? -- this is speculation on what is possible, I don't know the code of
udisks and if it uses this benefit)

As Lennart said, the feeding the CPU and IO to kernel will let it optimize
the load. Be grouping disk access or scheduling another process during IO
waits for the other. See below for one of my ideas on how to feed more
information to kernel.

> That's fine for "low priority" services out of the critical path for user
> interaction, but not for services we need as fast as possible.
>
> The best example I can think is the X server itself. It's pretty much the
> definition of the critical path. Everything started before the X server are
> those basic system components needed to start the X server, I understand in
> systemd that this core part would be a different target to the X part and a
> dep of it. Everything started after the X server is up are components of the
> user's session that aren't totally critical for them to login or use their
> computer.
>
> It's during the X startup that things get interesting. There are a few
> discreet phases of it.
>
> Firstly we have to spawn the X server, load the executable, map and
> relocate shared libraries, all of that jazz. You don't want anything else
> happening on the system alongside this, and you most certainly don't want
> every X client (Chromium/Firefox!) being loaded at the same time. The
> default behaviour of systemd seems to be to do just this :-(
>

Well, right now let's make clear that Firefox/Chromium or other X clients
won't be started by systemd as they are user session applications, and
systemd itself will just deal with system context (ie: up to GDM/KDM...).

But using your case, just like I said above, you really want that. You'd
chromium and X11 to be loaded at the same time, because the kernel will
handle the load and may load chromium executable while X11 is doing EDID
read, then may let the dynamic linker work while X11 asks for input device,
etc.

Also, ideally the socket activation lets you avoid cruft (unused
applications and services) being loaded. Let's say you have in your machine
Apache, MySQL, Squid for local development. None of these would be loaded,
because you never did a request to Apache that would make it to run, in turn
it would never talk to MySQL that would not start either. If you did no try
to access squid, it would not start either. Same for cups, avahi, ssh and
all dbus activated services... you can see that since less is activated,
less work is left to do.

> Secondly you have the period after the X server has "loaded" while
> it initializes but before it's "ready" to receive clients. In a perfect
> world this would be fast, but we all know that it isn't. In practice the X
> server takes countable seconds to do this work. This is a prime place in
> boot optimization to start other services alongside while X does its thang.
> But you wouldn't want to flood the system, as you don't want to delay X's
> startup, just use the spare resource to get more done. Can systemd discern
> "X has been spawned" and "X has initialized". Can systemd serialize services
> in this period (perhaps limited to one or two starting at a time)? Can it
> prioritize the order of those services in the X target?
>

It can't know. But it could be changed to be like udev-settle service and
force it artificially. But our approach is different and trying to fix the
actual issues are better, with X11, kernel or elsewhere.

But I do agree with you an a way to state our priorities would be awesome.
This is what Lennart said about a future way to feed it to kernel. I
suggested some hackish way some time ago:

http://lists.freedesktop.org/archives/systemd-devel/2011-March/001716.html

A hackish(?) solution would be to have a BootPriority=(True|False), set to
False by default and True for services we care most. Lower priority services
would be set to "background" priority in IO, CPU and others, then being
reset to actual values when systemd is notified. Problem is we need to
notify Systemd of that, as it's not a matter of just starting "gdm", but
actually gdm being in a "usable" state (defined by gdm itself) or desktop
being ready if users use autologin (like I do). This could also be stated as
"system is idle for X seconds", which would be monitored by systemd itself
and then no manual notification is required.

But its discussion is being delayed as at the moment things are going "fast
enough" without it (my userspace takes 1.2s in my Gentoo with SSD disk on a
MacBook Pro).

Lastly there's the processes you'd start after X is up and services are
> connecting to it; again I assume this would be a different target - how
> would systemd know that X, and the critical X services/processes are fully
> up and that it can begin starting these. The important thing here is user
> latency - the startup of these services need to not noticeably impact system
> performance - not as simple as prioritization because the service itself may
> well need a different priority once its up. Any thoughts here?
>

During LPC last year Lennart, Arjan and others discussed this. Arjan
suggested doing this artificial throttling to not trigger kernel fork-bomb
protections. Lennart argued that this should not happen given the number of
things we start and if that's the case, it should be fixed in the kernel.
Given that MeeGo is now moving to systemd, I wonder if this problem happens
or if it's so small it was ignored.   During this discussion we wondered
about changing kernel parameters, like making the system less interactive
during startup to speed up boot, reverting to fully interactive later, but
it proved to be more work than what it was worth.

Hope it help to clarify :-)

Regards,

-- 
Gustavo Sverzut Barbieri
http://profusion.mobi embedded systems
--------------------------------------
MSN: barbieri at gmail.com
Skype: gsbarbieri
Mobile: +55 (19) 9225-2202
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/systemd-devel/attachments/20110509/655655be/attachment.htm>