<div dir="ltr"><div class="gmail_default" style="font-family:comic sans ms,sans-serif"><div style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;font-size:12pt;line-height:inherit;font-family:Calibri,Arial,Helvetica,sans-serif,serif,EmojiFont;vertical-align:baseline;color:rgb(0,0,0)">Hi Folks,<br><br><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit">I am finding anomalous behavior when I am trying to run dhclient process inside my docker container in vanilla Ubuntu 16.04 host. The service gets into "deactivating" state and is stuck forever. In the mail I have attached a minimalistic reproduction of the issue seen.</div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><br></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit">Working logic:</div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><ul><li>There is a sample trial@.service script which invokes the `trial` binary with the option passed to the systemd service via @ option</li><li>The valid options are sleep and dhclient_<interface_name></li><li>The binary either invokes a long-lived sleep process or dhclient process on the said interface_name based on the input</li><li>The binary then spawns `kill_trial.sh` script. The script sleeps for 20 seconds and kills the parent `trial` binary. The kill signal is SIGKILL in the trial example. In the real-world, this can be a SIGSEGV indicating a crash in the parent process.</li><li>If the trial binary was started for sleep process things work fine and service goes into "failed" state as expected</li><li>However, in case of dhclient, the service is stuck in "deactivating" state if the underlying host OS is Ubuntu 16.04. This works well if the host is running Ubuntu 20.04.</li><li>We have kept TimeoutStopSec to infinity, because in real-word deployments, the core collection post a crash takes varying time depending on the memory config on the host.</li></ul></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><br></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit">Steps to reproduce</div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"># tar -xf minimal_repro.tar -C minimal_repro/</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"># cd <span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:Calibri,Arial,Helvetica,sans-serif,serif,EmojiFont;vertical-align:baseline"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit">minimal_repro/</span></span></span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"># docker build -t trial .</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"># docker rm -f trial</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"># docker run -it -d --net=host --privileged -v /sys/fs/cgroup:/sys/fs/cgroup:ro --name trial trial</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"># docker exec -it trial bash</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><br></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"># systemctl start trial@dhclient_eth1.service</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><br></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"># #Keep monitoring trial@dhclient_eth1.service -- issue should be seen within 20-30 seconds on Ubuntu 16.04 host</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><br></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"># systemctl status trial@dhclient_eth1.service</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit">● trial@dhclient_eth1.service - Trial</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit">     Loaded: loaded (/etc/systemd/system/trial@.service; static; vendor preset: enabled)</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit">     Active: deactivating (stop-sigterm) (Result: signal) since Mon 2021-06-07 13:19:12 UTC; 1min 11s ago</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit">    Process: 55 ExecStartPre=/bin/bash /etc/systemd/system/trial_service_script.sh pre_start dhclient_eth1 (code=exited, status=0/SUCCESS)</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit">    Process: 56 ExecStart=/bin/bash /etc/systemd/system/trial_service_script.sh start dhclient_eth1 (code=killed, signal=KILL)</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit">   Main PID: 56 (code=killed, signal=KILL)</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit">      Tasks: 0 (limit: 38590)</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit">     Memory: 588.0K</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit">     CGroup: /docker/903fca0cee1387b7c2113a36ee5efdb3a25edd1e60584fe5da5d0c5b5ffd8241/system.slice/system-trial.slice/trial@dhclient_eth1.service</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><br></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"># #NOTE: `Active: deactivating` -- in stuck state</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"># #Running `systemctl daemon-reload` forces the service to go to failed state</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><br></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"># systemctl start trial@sleep.service</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><br></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"># #Keep monitoring trial@sleep.service -- would be killed in 20-30 seconds and goes into failed state as expected</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><br></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"># # systemctl status trial@sleep.service</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit">● trial@sleep.service - Trial</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit">     Loaded: loaded (/etc/systemd/system/trial@.service; static; vendor preset: enabled)</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit">     Active: failed (Result: signal) since Mon 2021-06-07 13:38:19 UTC; 21s ago</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit">    Process: 113 ExecStartPre=/bin/bash /etc/systemd/system/trial_service_script.sh pre_start sleep (code=exited, status=0/SUCCESS)</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit">    Process: 114 ExecStart=/bin/bash /etc/systemd/system/trial_service_script.sh start sleep (code=killed, signal=KILL)</span></div><div style="margin:0px;padding:0px;border:0px;font:inherit;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit">    Process: 129 ExecStopPost=/bin/bash /etc/systemd/system/trial_service_script.sh post_stop sleep (code=exited, status=0/SUCCESS)</span></div><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit">   Main PID: 114 (code=killed, signal=KILL)</span><br></div><div style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;font-size:12pt;line-height:inherit;font-family:Calibri,Arial,Helvetica,sans-serif,serif,EmojiFont;vertical-align:baseline;color:rgb(0,0,0)"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"><br></span></div><div style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;font-size:12pt;line-height:inherit;font-family:Calibri,Arial,Helvetica,sans-serif,serif,EmojiFont;vertical-align:baseline;color:rgb(0,0,0)"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:Calibri,Arial,Helvetica,sans-serif,serif,EmojiFont;vertical-align:baseline">Please advise on what can help us in alleviating the issue. </span><br></span></div><div style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;font-size:12pt;line-height:inherit;font-family:Calibri,Arial,Helvetica,sans-serif,serif,EmojiFont;vertical-align:baseline;color:rgb(0,0,0)"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"><br></span></div><div style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;font-size:12pt;line-height:inherit;font-family:Calibri,Arial,Helvetica,sans-serif,serif,EmojiFont;vertical-align:baseline;color:rgb(0,0,0)"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:12pt;line-height:inherit;font-family:calibri,arial,helvetica,sans-serif,serif,EmojiFont;vertical-align:baseline">Thanks,</span><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:"Courier New",monospace,serif,EmojiFont;vertical-align:baseline;color:inherit"><br></span><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:12pt;line-height:inherit;font-family:calibri,arial,helvetica,sans-serif,serif,EmojiFont;vertical-align:baseline">Aravindhan</span></div><div style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;font-size:12pt;line-height:inherit;font-family:Calibri,Arial,Helvetica,sans-serif,serif,EmojiFont;vertical-align:baseline;color:rgb(0,0,0)"><span style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:12pt;line-height:inherit;font-family:calibri,arial,helvetica,sans-serif,serif,EmojiFont;vertical-align:baseline"><br></span></div></div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><span style="font-family:'comic sans ms',sans-serif">Regards,</span><br></div><div><font face="comic sans ms, sans-serif">Aravindhan Krishnan...</font></div></div></div></div></div>