[systemd-devel] TimeoutStopSec is ignored (regression)

[Michael Chapman]

On Mon, 1 Dec 2014, David Herrmann wrote:
> Hi
>
> On Sat, Nov 29, 2014 at 12:35 PM, Ross Lagerwall
> <rosslagerwall at gmail.com> wrote:
>> Hi,
>>
>> On recent versions of systemd, unit_kill_context doesn't set
>> wait_for_exit to true which means that service_enter_signal sends
>> SIGTERM, immediately moves into stop-sigkill and sends SIGKILL, ignoring
>> TimeoutStopSec and often killing processes without giving them a chance
>> to cleanup.
>>
>> Reverting the following change, fixes the problem:
>>
>> commit 1baccdda2e954214e0c5463d6ed8f06009b33c41
>> Author: Lennart Poettering <lennart at poettering.net>
>> Date:   Wed Feb 5 02:22:11 2014 +0100
>>
>>     core: don't wait for non-control/non-main processes when killing processes on the host either
>>
>>     Since the current kernel cgroup notification logic is easily confused by
>>     existing subgroups, let's do the same thing as in containers before. and
>>     just not wait for non-control and non-main processes.
>>
>>     This should be corrected as soon as we have sane cgroup notifications
>>     from the kernel.
>
> The commit-message and the comment it adds should answer your
> question: The kernel cgroup API does not allow us to wait for
> non-control processes. That is, we still honor TimeoutStopSec and
> friends if we have to wait for the main-process and/or control process
> (in those cases, wait_for_exit is still set to true). However, if
> there are other processes remaining in the cgroup, we now ignore it.
> See the commit you mentioned for an explanation.

What specifically would happen if wait_for_exit were kept true for other 
processes in the cgroup?

As far as I can see they would continue to be watched for SIGCHLD (since 
unit_watch_all_pids should have been previously called on the unit). PID 1 
may or may not get SIGCHLD for them, depending on whether they got 
reparented before they exited. Each time systemd gets a SIGCHLD, it can 
use unit_tidy_watch_pids to check the unit's entire PID list to see which 
ones are still present.

So at best we see the PIDs go away one by one in the cgroup, and we know 
when it's empty ourselves. At worst we don't see the last PID's SIGCHLD, 
so we have to wait the entire TimeoutStopSec interval before discovering 
that the cgroup is empty.

So I must be missing something important here, since everyone is stating 
emphatically that this is unsolveable until cgroup empty notifications are 
fixed. The only issue I can think of is that PIDs may be reused before the 
TimeoutStopSec interval completes.

- Michael