[systemd-devel] GuessMainPID=no required to make daemon reload work

[Umut Tezduyar Lindskog]

Hi,


On Sun, May 4, 2014 at 11:19 PM, Gerd v. Egidy <lists at egidy.de> wrote:
> Hi,
>
> I'm one of the developers of the Icinga monitoring system. We want to provide
> a .service file for our monitoring daemon with the next major release (Icinga
> 2).
>
> Due to technical reasons, the daemon can't reload it's configuration files
> within one process. So we implement reload like this:
>
> 1. daemon process detects a SIGHUP or reload request on a control channel
> 2. a new process is forked & execed with a special commandline argument
> indicating a reload
> 3. the old daemon continues it's normal work
> 4. the new process loads and checks the configuration (this can be time
> consuming)
> 5. a) on config errors the new process terminates. the old daemon detects the
> SIGCHLD, logs error messages and so on
> 5. b) when the config is ok the new process sends a SIGTERM to the old daemon
> 6. the old daemon shuts down due to the SIGTERM
> 7. the new process detects when the old daemon is gone, activates it's config
> and starts it's work
>
> This is much better than a regular restart:
> a) the time consuming config check is done in background
> b) service is not interrupted if the new config is invalid
>
> But this scheme seems not to work well with systemd: After reload systemd
> thinks the daemon has ended and declares the whole service "inactive (dead)",
> usually killing the remaining daemon.
>
> After diving into the systemd manpages and trying some options, I found that I
> could make it work with GuessMainPID=no:
>
> [Unit]
> Description=Icinga host/service/network monitoring system
> After=syslog.target postgresql.service mariadb.service
>
> [Service]
> Type=forking
> EnvironmentFile=/etc/sysconfig/icinga2
> ExecStart=/usr/sbin/icinga2 -c ${ICINGA2_CONFIG_FILE} -d -e
> ${ICINGA2_ERROR_LOG} -u ${ICINGA2_USER} -g ${ICINGA2_GROUP}
> ExecReload=/bin/sh -c '/bin/kill -HUP `cat
> ${ICINGA2_STATE_DIR}/run/icinga2/icinga2.pid`'
> GuessMainPID=no
>
> [Install]
> WantedBy=multi-user.target
>
> Systemd seems to successfully detect the daemon pids and some short tests with
> systemd-208-16.fc20.x86_64 were successful. But I'm not sure about the
> downsides of GuessMainPID=no or if there are some problems with older versions
> of systemd or if future changes in systemd will pose problems.
>
> Can you describe the downsides of GuessMainPID=no more specific than "failure
> detection and automatic restarting of a service will not work reliably"?

One down side I can mention is for the fault recovery. If your
application crashes and if systemd is configured to restart it
(Restart=), then systemd has to know which process it should watch so
it can restart it.

How about having 1 process as the interface to systemd and that
process is forking it self as the "old" and "new" processes (as you
have named)?

Umut

>
> Can you recommend a better alternative for the reload scheme outlined above?
>
> I'd prefer if I could tell systemd about the pidfile. Systemd should then
> either automatically detect updates of the pidfile (e.g. via inotify) or at
> least re-read it if it thinks the service has changed state.
>
> Kind regards,
>
> Gerd
>
> _______________________________________________
> systemd-devel mailing list
> systemd-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/systemd-devel