[systemd-devel] systemctl [en|dis]able weirdness + reload (writes /run/nologin)

[Colin Guthrie]

'Twas brillig, and Colin Guthrie at 13/01/14 15:33 did gyre and gimble:
> 'Twas brillig, and Colin Guthrie at 13/01/14 15:08 did gyre and gimble:
>> 'Twas brillig, and Colin Guthrie at 13/01/14 11:30 did gyre and gimble:
>>> 'Twas brillig, and Colin Guthrie at 13/01/14 11:16 did gyre and gimble:
>>>> 2. This is the much odder part of the problem I'm seeing. The call to
>>>> daemon_reload() at the end of the enable_unit() seems to trigger some
>>>> kind of broken daemon reload that puts things into a bad state,
>>>> including a stale /run/nologin file.
>>>>
>>>> I'm not sure WHY this does this, but it's very reliably reproducible. I
>>>> have a native sysvinit script called numlock. All I need to do to
>>>> trigger the bad state is "systemctl disable numlock". After the call,
>>>> the systemd daemon is reloaded and it goes into this bad state
>>>> completely with /run/nologin file.
>>>>
>>>> If I comment out call or use --no-reload, then all is well. If I call
>>>> "systemctl daemon-reload" on it's own, all seems well. It just seems to
>>>> be this reload call specifically at the end of enable_unit() that
>>>> triggers the bad state.
>>>>
>>>>
>>>>
>>>> I'm going to try reverting some of the patches I have applied to see
>>>> where I get with things, as I see Zbigniew backed a few out of fedora
>>>> due to freeze rules, but I did also see some threads from Zbigniew about
>>>> the whole /run/nologin, so I suspect he may be interested in this.
>>>
>>> I reverted the same patches that were reverted in Fedora so our builds
>>> should be quite similar.
>>>
>>> I really hope fedora has this same issue otherwise my debugging just got
>>> more confusing.
>>>
>>> Zbigniew can you reproduce this on F20?
>>
>> It seems to be specifically related to chkconfig. If I shell out instead
>> to something different (e.g. "whoami") all runs fine.
>>
>> I'm wondering if it's something related to semi-systemd stuff supported
>> in our chkconfig... Perhaps our patches are out of date compared to
>> fedora...
>>
>> Still hunting :)
> 
> OK, so it seems that chkconfig will these days notify systemd to reload
> itself.
> 
> static void reloadSystemd(void) {
>     if (systemdActive())
>         system("systemctl daemon-reload > /dev/null 2>&1");
> }
> 
> For whatever reason, doing this in the forked off process AND in systemd
> itself leads to some kind of race.
> 
> Perhaps this happens if two reload operations come in in very quick
> succession, or perhaps the use of system() (and it's subsequent fork) in
> chkconfig just somehow messes up our signal handling in systemctl?
> 
> Either way, commenting this out in systemctl avoids the problem.
> 
> I would suggest three possible fixes:
> 
> 
> 1. Find out why this is racey and fix it.
> 2. Add an option to chkconfig to disable the reload.
> 3. Just drop the reload completely from chkconfig.
> 
> I would suspect that the option route is the best way forward but
> chkconfig will bail out of unsupported options so we should either use
> an ENV var or make sure systemd and chkconfig are updated in lockstep.
> 
> Thoughts?


OK, I've added both an env var and a new option to chkconfig and then
call chkconfig with the appropriate arguments from systemctl.

Attached are my two patches (keep in mind that the earlier patch to
avoid bogus warnings about [Install] sections is still valid too.

Seems to solve the issue for me (at least in my test case).

Cheers!

Col


-- 

Colin Guthrie
gmane(at)colin.guthr.ie
http://colin.guthr.ie/

Day Job:
  Tribalogic Limited http://www.tribalogic.net/
Open Source:
  Mageia Contributor http://www.mageia.org/
  PulseAudio Hacker http://www.pulseaudio.org/
  Trac Hacker http://trac.edgewall.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0514-systemctl-Ensure-the-no-reload-and-no-redirect-optio.patch
Type: text/x-patch
Size: 1784 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/systemd-devel/attachments/20140113/00ddb5ce/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: chkconfig-1.3.61-no-systemd-reload.patch
Type: text/x-patch
Size: 3016 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/systemd-devel/attachments/20140113/00ddb5ce/attachment-0001.bin>