[systemd-devel] Starting configurable set of services first

[Lennart Poettering]

On Mon, 27.10.14 17:06, Umut Tezduyar Lindskog (umut at tezduyar.com) wrote:

> >> Should this be something we should support natively by systemd?
> >
> > As discussed at th systemd hackfest: I am a bit conservative about
> > this as it introduces plenty chance for deadlocks, where services
> > might trigger/request some other unit but we'd delay it until the
> > later stage...
> >
> > I think the implementation you chose is actually pretty good. I am not
> > sure though that we should do this upstream. I mean, I really would
> > prefer if we'd dump as much work as possible on the IO elevator and
> > CPU scheduler, and then adjust the priorities of it to give hints what
> > matters more. Trying to second-guess the elevator and scheduler in
> > userspace feels a bit like chickening out to me, even though I am sure
> > that it might be something that one has to do for now, in the real
> > world...
> 
> I am not agreeing on this. Once you fork the process, it will always
> get some CPU even though you play with cpu.shares, sched_latency_ns,
> sched_min_granularity_ns. My goal is not forking it at all until high
> priority services are activated. Just like Before=, After=.

That sounds like a limitation of the kernel's scheduler and the "cpu"
cgroup controller, no?

If there's no way to ensure that a process we fork() off gets exactly
zero CPU until the point where otherwise nothing needs to run, then
that's something to fix in the kernel, no?

I really would like to stay away from working around kernel
limitations in userspace if we can. In particular since doing this in
userspace is an invitation of all kinds of deadlocks where we would
refuse activating certain low-priority units even though high-priority
ones are waiting for it.

Note that by adding a sufficeint number of ordering deps for systemd
units, and making sure all services use sd_notify() late enough to
signal full completion of their initializatin you can already
completely remove any possible parallel execution in systemd. i.e. for
all possible combination of two units that could be run at the same
time you could add a manual dependency (of course always making sure
manually that you keep thigns cycle-free), which systemd would then
execute strictly linearly, since it will adhere to those deps...

> > There's one change I'd really like to see done though in systemd, that
> > might make things nicer for you. Currently, it's undefined in systemd
> > which job is dispatched first, if multiple jobs can be executed. That
> > means when we are about to fork off a number of processes there's no
> > way to control which one gets forked off first. I'd be willing to
> > merge a patch that turns this into a prioq, so that some priority
> > value can be configured (or automatically derived) for each unit, and
> > the one with the highest priority would win, and be processed
> > first. This would not provide you with everything what you want, but
> > would make things a bit nicer when we dump all possible work on the
> > scheduler/elevator, because after all we cannot really dump all work
> > at the same time, and hence should at least give you control in which
> > order to dump it, if you follow what I mean.
> 
> I have understood your propose with the exception of one thing. When
> do we start dispatching low priority jobs? When the high priority jobs
> are dispatched/forked or when the high priority jobs are
> dispatched/activated?

We'd simply process all queued jobs that are runnable in the order of
their priority. This means:

1) we'd still dispatch only one job at a time (i.e. fork() off one
   process at a time) 
2) we'd dispatch all jobs as quickly as possible, as soon as they are
   runnable
3) If two jobs are runnable at the same time, we'd dispatch the one
   with the higher priority first.

Lennart

-- 
Lennart Poettering, Red Hat