[systemd-devel] Reason for setting runqueue to IDLE priority and side effects if this is changed?

[Andrei Borzenkov]

В Wed, 15 Jul 2015 17:30:53 +0000
"Hoyer, Marko (ADITG/SW2)" <mhoyer at de.adit-jv.com> пишет:

> Hi all,
> 
> jumping from systemd 206 to systemd 211 we were faced with some issue, which are finally caused by a changed main loop priority of the job execution. 
> 
> Our use case is the following one:
> ----------------------------------
> While we are starting up the system, a so called application starter is bringing up a set of applications at a certain point in a controlled way by requesting systemd via dbus to start respective units. The reason is that we have to take care for applications internal states as synchronization points and for better meeting timing requirements of certain applications in a generic way. I told the story once before in another post about watchdog observation in state "activating". However ...
> 
> Up to v206, the behavior of systemd was the following one:
> ----------------------------------------------------------
> - the starter sends out a start request of a bench of applications (he requests a sequence of unit starts)

If you want to control order of execution yourself, why do you not wait
for each unit to start before submitting next request?

> - it seems that systemd is working of the sequence exactly as it was requested one by one in the same order
> 
> Systemd v211 shows a different behavior:
> ----------------------------------------
> - the starter sends out a bench of requests
> - the job execution of the first job is significantly delayed (we have a system under stress, high CPU load at that time)
> - suddenly, system starts working of the jobs but now in reverse order
> - depending on the situation, it might happen that a complete bench of scheduled jobs are reverse ordered, sometimes two or more sub benches of jobs are executed in the reverse order (the jobs in each bench are reverse ordered)
> 
> I found that the system behavior with systemd v206 was only accidently the expected one. The reason was that in this version the run queue dispatching was a fixed part of the main loop located before the dispatching of the events. This way, it gained higher priority than the dbus request handling. One jobs was requested via dbus. Once the dbus job request was dispatched, it's been worked off immediately the next round of the mainloop. Then the next dbus request was dispatched and so on ...
> 
> Systemd v211 added the running queue as deferred event source to the event handling with priority IDLE. So dbus requests are preferred over job execution. The reverse order effect is simply because the run queue is more a stack than a queue. All of the observed behavior could be explained I guess.
> 
> So long story in advance. I've now two questions:
> - Am I causing any critical side effects when I'm increasing the run queue priority so that it is higher than the one of the dbus handling (which is NORMAL)? First tests showed that I can get back exactly the behavior we had before with that.
> 
> - Might it still happen that situations are happening where the jobs are reordered even though I'm increasing the priority?
> 
> - Is there any other good solution ensuring the order of job execution?
> 

systemd never promised anything about relative order of execution
unless there is explicit dependency between units. So good solution is
to put dependency in unit definitions. submit the whole bunch at once
and let systemd sort the order.