[systemd-devel] syslog makes impossible to enter emergency mode

[Andrey Borzenkov]

В Tue, 26 Nov 2013 02:27:00 +0100
Lennart Poettering <lennart at poettering.net> пишет:

> On Tue, 26.11.13 01:00, Lennart Poettering (lennart at poettering.net) wrote:
> 
> > 
> > On Sun, 24.11.13 22:36, Andrey Borzenkov (arvidjaar at gmail.com) wrote:
> > 
> > > Interesting case (https://bugzilla.novell.com/show_bug.cgi?id=852021).
> > > Systemd enters emergency due to failed mount. At the same time syslog
> > > socket triggers syslog.service. Due to implicit Requires on
> > > basic.target which Requires sysinit.target which conflicts with
> > > emergency.{service,target} syslog.service tries to start basic.target
> > > (it is not there yet ...) which apparently kills emergency shell.
> > 
> > This was probably introduced by
> > 80cfe9e163b1c92f917e0a5e053b148fca790677.
> > 
> > I figure we should find something in the middle of
> > OnActivationIsolate=yes and OnActivationIsolate=no. i.e. make use of
> > the "replace-irreversible" job mode which will allow the emergency jobs
> > to be queued without being reversible by later requests until they are
> > finished or explicitly flushed out with "systemctl cancel".
> > 
> > I figure I'll replace OnActivationIsolate=yes by OnActivationMode= which
> > takes the full range of job modes, and then turn OnActiveIsolate= into a
> > hidden compat switch
> 
> Done.
> 
> Lennart
> 

Does not work. Quoting man page: "creates jobs that cannot be reversed
UNLESS THEY FINISHED or are explicitly canceled". This is exactly what
happens here - emergency.target job is already finished, then syslog
kicks in and tries to starts basic.target => sysinit.target which kills
emergency service.

Nov 30 09:48:57 linux-cor4 systemd[1]: Dependency failed for Local File Systems.
Nov 30 09:48:57 linux-cor4 systemd[1]: Triggering OnFailure= dependencies of local-fs.target.
Nov 30 09:48:57 linux-cor4 systemd[1]: Trying to enqueue job emergency.target/start/replace-irreversibly
...
Nov 30 09:48:58 linux-cor4 systemd[1]: Job emergency.service/start finished, result=done
Nov 30 09:48:58 linux-cor4 systemd[1]: Started Emergency Shell.
Nov 30 09:48:58 linux-cor4 systemd[1]: Starting Emergency Mode.
Nov 30 09:48:58 linux-cor4 systemd[1]: emergency.target changed dead -> active
Nov 30 09:48:58 linux-cor4 systemd[1]: Job emergency.target/start finished, result=done
Nov 30 09:48:58 linux-cor4 systemd[1]: Reached target Emergency Mode.
...
Nov 30 09:48:58 linux-cor4 systemd[1]: Incoming traffic on syslog.socket
Nov 30 09:48:58 linux-cor4 systemd[1]: Trying to enqueue job rsyslog.service/start/replace
...
Nov 30 09:48:58 linux-cor4 systemd[1]: Installed new job emergency.service/stop as 192
Nov 30 09:48:58 linux-cor4 systemd[1]: Installed new job emergency.target/stop as 193

This is obvious race condition - some people reported success some -
not. I can reliably reproduce failure :)

And it does not look like using isolate would actually change anything
here ... we'd be in the same situation.

Short term solution is to make emergency.service conflict with
sysylog.socket, but this potentially can be triggered by any other
socket so it obviously does not scale.

So to repeat the question - should not *default* service dependency be
Requisite instead of Requires?