[systemd-devel] Parallel startup with sockets and without killing the machine?

[Lennart Poettering]

On Mon, 09.05.11 09:10, Scott James Remnant (scott at netsplit.com) wrote:

> This seems to assume that it's a free cost to start a service or process and
> have it block on a socket. But that isn't true, to get to the point where it
> connects to the socket, that service or process first has to be loaded from
> disk, its shared libraries mapped and relocated, it probably has to do a
> bunch of initialization including reading its own configuration files,
> etc.
>
> That's fine for "low priority" services out of the critical path for user
> interaction, but not for services we need as fast as possible.

We can now boot a resonably complete GNOME userspace in less than
1s. Given that that is the *total* these days, I really don't
understand why you are concerned about the startup speed of individual
service, and fear timeouts there.

> The best example I can think is the X server itself. It's pretty much the
> definition of the critical path. Everything started before the X server are
> those basic system components needed to start the X server, I understand in
> systemd that this core part would be a different target to the X part and a
> dep of it. Everything started after the X server is up are components of the
> user's session that aren't totally critical for them to login or use their
> computer.

X itself is really fast these days. While there's room for improvement
it's far from the biggest speed bump right now. (LVM is)

> It's during the X startup that things get interesting. There are a few
> discreet phases of it.

I don't find X very interesting here at all.

> Firstly we have to spawn the X server, load the executable, map and relocate
> shared libraries, all of that jazz. You don't want anything else happening
> on the system alongside this, and you most certainly don't want every X
> client (Chromium/Firefox!) being loaded at the same time. The default
> behaviour of systemd seems to be to do just this :-(

Well, there's more than 30 years research in modern CPU and IO scheduler
technology for the kernel. Instead of second guessing those from
userspace I'd just entrust them that they pick something like the
optimal order. And if they don't, then go and optimize them for the new
workloads, don't bypass them by scheduling things in userspace.

Of course, we could microoptimize things and pass along to the kernel
scheduler useful priority data gathered from our dep tree. But right now
I am more interested in harvesting the easier and bigger improvements we
can do. Like getting rid of shell scripts on the full distro, and making
sure the initrd isn't delayed by DDC probing. And after we have done
that, we can focus on passing on prio information.

> Secondly you have the period after the X server has "loaded" while
> it initializes but before it's "ready" to receive clients. In a perfect
> world this would be fast, but we all know that it isn't. In practice the X
> server takes countable seconds to do this work. This is a prime place
> in

That is not what I am seeing here.

> boot optimization to start other services alongside while X does its thang.
> But you wouldn't want to flood the system, as you don't want to delay X's
> startup, just use the spare resource to get more done. Can systemd discern
> "X has been spawned" and "X has initialized". Can systemd serialize services
> in this period (perhaps limited to one or two starting at a time)? Can it
> prioritize the order of those services in the X target?

No, we emphasize parallelization. We want the IO and the CPU scheduler
of the kernel to schedule things for us. They are much better at these
things. If we tried this in userspace most of the time the CPU/IO would
be less than fully utilized.

> Lastly there's the processes you'd start after X is up and services are
> connecting to it; again I assume this would be a different target - how
> would systemd know that X, and the critical X services/processes are fully
> up and that it can begin starting these. The important thing here is user
> latency - the startup of these services need to not noticeably impact system
> performance - not as simple as prioritization because the service itself may
> well need a different priority once its up. Any thoughts here?

Ignoring that X currently is not socket activatable I see no issue here
at all. 

Also, note that launchd on MacOS does the same thing (including spawning
X like this for X clients). They build their whole OS like this. And it
works really really really well there. All experience with launchd and
systemd tells the same story: socket activation is awesome. Plain awesome.

We boot userspace in < 1s. Really. I mean, doesn't this give a pretty
strong signal that things are working perfectly with the approach we
chose?

Lennart

-- 
Lennart Poettering - Red Hat, Inc.