[systemd-devel] systemd and smartd daemon

[Al Lau]

As a test, the "/usr/sbin/smartd $smartd_opts" is invoked from the command
line and the daemon is forked and stayed up as expected.  By default, the
/usr/sbin/smartd daemonize.

# cat /etc/sysconfig/smartmontools
smartd_opts="--interval=3600 --configfile=/etc/smartd.conf"

>From the /var/log/messages, the SIGTERM signal is sent to the forked
process.  Like to know if this is something that can be worked around.

Thanks,
-Al

On Thu, Apr 2, 2015 at 11:30 AM, Lennart Poettering <lennart at poettering.net>
wrote:

> On Thu, 02.04.15 10:43, Al Lau (lauatic at gmail.com) wrote:
>
> > Hi,
> >
> > I am seeking help and advise on troubleshooting the starting of the
> smartd
> > daemon.  The /usr/sbin/smartd comes from the smartmontools version 6.3.
> >
> > The /usr/lib/systemd/system/smartd.service file looks like this
> >
> > # cat /usr/lib/systemd/system/smartd.service
> > [Unit]
> > Description=Self Monitoring and Reporting Technology (SMART) Daemon
> > Documentation=man:smartd(8) man:smartd.conf(5)
> > After=syslog.target
>
> This line is not necessary. Nowadays all services are started with
> /dev/log available. Please remove.
>
> >
> > [Service]
> > Type=forking
> > PIDFile=/run/smartd.pid
> > ExecStartPre=/bin/rm -f /run/smartd.pid
>
> systemd removes PID files listed in PIDFile= automatically these days,
> this line is unnecessary.
>
> > EnvironmentFile=/etc/sysconfig/smartmontools
> > ExecStart=/usr/sbin/smartd $smartd_opts
>
> I assume that your $smartd_opts includes any necessary switches to
> tell smartd to daemonize?
>
> > ExecReload=/bin/kill -HUP $MAINPID
> > StandardOutput=syslog
>
> This line is not necessary, systemd since quite some time connects all
> daemon stdout/stderr to the journal anyway.
>
> > [Install]
> > WantedBy=multi-user.target
> >
> > When "systemctl start smartd.service" is called, the process forks into a
> > daemon.  The problem I'm seeing is that the forked process received a
> > SIGTERM signal and exited.  How do I resolve this so that the forked
> > process would not get terminated.
> >
> > # systemctl status smartd.service
> > smartd.service - Self Monitoring and Reporting Technology (SMART) Daemon
> >    Loaded: loaded (/usr/lib/systemd/system/smartd.service; enabled)
> >    Active: failed (Result: timeout) since Thu 2015-04-02 17:19:49 GMT;
> 4min
> > 39s ago
> >      Docs: man:smartd(8)
> >            man:smartd.conf(5)
> >   Process: 7456 ExecStart=/usr/sbin/smartd $smartd_opts (code=exited,
> > status=0/SUCCESS)
> >   Process: 7454 ExecStartPre=/bin/rm -f /run/smartd.pid (code=exited,
> > status=0/SUCCESS)
> >  Main PID: 1075 (code=exited, status=0/SUCCESS)
> >
> > Apr 02 17:18:20 007 smartd[7456]: Device: /dev/bus/0 [megaraid_disk_19],
> > opened
> > Apr 02 17:18:20 007 smartd[7456]: Device: /dev/bus/0 [megaraid_disk_19],
> > [TOSHIBA  MG03SCA300       5702], lu ....00 TB
> > Apr 02 17:18:20 007 smartd[7456]: Device: /dev/bus/0 [megaraid_disk_19],
> is
> > SMART capable. Adding to "monitor" list.
> > Apr 02 17:18:20 007 smartd[7456]: Monitoring 0 ATA and 12 SCSI devices
> > Apr 02 17:18:21 007 systemd[1]: PID file /run/smartd.pid not readable
> > (yet?) after start.
> > Apr 02 17:18:21 007 smartd[7463]: smartd has fork()ed into background
> mode.
> > New PID=7463.
> > Apr 02 17:19:49 007 systemd[1]: smartd.service operation timed out.
> > Terminating.
> > Apr 02 17:19:49 007 smartd[7463]: smartd received signal 15: Terminated
> > Apr 02 17:19:49 007 systemd[1]: Failed to start Self Monitoring and
> > Reporting Technology (SMART) Daemon.
> > Apr 02 17:19:49 007 systemd[1]: Unit smartd.service entered failed state.
> > Hint: Some lines were ellipsized, use -l to show in full.
> > #
> >
> > To verify, I take the "/usr/sbin/smartd $smartd_opts" and run it from the
> > command line.  The smartd daemon forks and the daemon process stays up as
> > expected.
>
> The output above really suggests that the daemonization code in smartd
> is broken?
>
> A sysv daemon should do the following when daemonizing:
>
> 1) fork twice, exit in the middle child
> 2) in the grandchild (the main daemon process), set everything up,
>    including writing the PID file
> 3) exit in the parent
>
> It is important that 3) does not happen before 2) is complete and the
> PID file exists. The output above suggests that smartd doesn't get
> this right, and sometimes writes the PID file after exiting in the parent.
>
> This is a bug that needs to be fixed in smartd, and it is not specific
> to systemd's way of watching daemons. With this broken things like
> this on a pure sysv system are racy too:
>
>      /etc/init.d/smartd start ; /etc/init.d/smartd stop
>
> (because the PID file might not exist yet when the stop command tries
> to read it to kill the daemon).
>
> Also see daemon(7) for a more detailed overview.
>
> Anyway, this really smells like a daemon bug, not a bug in your unit
> file, and it needs to be fixed in the daemon.
>
> (An alternative is to not let the daemon daemonize, but tell it to
> stay in the foreground and then use the default Type=simple. However,
> this means that systemd will not wait for the daemon to be fully
> initialized, which might or might not be an issue, depending on the
> daemon, for example whether it needs to be accessible via IPC or so)
>
> Lennart
>
> --
> Lennart Poettering, Red Hat
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/systemd-devel/attachments/20150402/02f780dc/attachment-0001.html>