[systemd-devel] Complex supervision structures/delegating watchdog?

[Holger Hans Peter Freyther]

On Mon, Feb 16, 2015 at 11:21:53AM +0100, Lennart Poettering wrote:
> 
> Is your pppd daemon itself also a systemd service?
> 
> What precisely does "monitor" consist of for this case?

Yes, the pppd is invoked by systemd as a service. What I am
doing right now is:

link.service:
ExecStart=pppd call uplink ... nodetach

/etc/ppp/ip-up.d/NN-linkmon:
linkmon -i $PPP_IFACE -d 8.8.8.8 -p `cat /run/${PPP_IFACE}.pid` &


The monitoring right now involves simple ICMP request and
depending on the outcome I change the metric of one of the
default routes. So in case some amount of packet loss is
reached the linkmon will SIGKILL the pid provided.

> We have watchdog support already, with sd_notify(0, "WATCHDOG=1"), and
> WatchdogSec=. But that requires you to run your pppd as a service of
> its own, to be useful.

I thought the sd_notify is only possible by the "main"
application that got started? E.g. in the above case the
linkmon would be a child of pppd. My application wouldn't
run until pppd has setup the link. This means I would need
to configure a high enough timeout to cope with a potential
bigger set-up time.


One nice thing for an external watchdog started by the
NN-linkmon script would be that it would be under pid1
control (e.g. if it is crashing, the other service would
be taken down by systemd), I could have different privileges
for the monitoring system (currently at least one part of
it must be able to send a kill to a parent process).

Another neat feature would be if applications could
communicate some extra (custom) status to systemd. E.g.
in the case of pppd to indicate the state of the link-setup,
for something like our BTS process to indicate if it is
currently broadcasting or muted.

Thanks for listening
	holger