[systemd-devel] GuessMainPID=no required to make daemon reload work

[Brandon Black]

On Fri, May 16, 2014 at 9:38 AM, Lennart Poettering
<lennart at poettering.net>wrote:

> On Thu, 15.05.14 20:57, Brandon Black (blblack at gmail.com) wrote:
>
> > On Thu, May 15, 2014 at 6:40 PM, Lennart Poettering
> > <lennart at poettering.net>wrote:
> > >
> > > But again, it's generally not a good idea to keep file locks for a
> > > longer period of time, much as with mutexes... You don't want to make
> > > other apps which try to get an atomic view on the file hang for long.
> >
> > If they don't want to hang, they don't have to.  There are nonblocking
> > calls to check for a conflicting advisory fcntl() lock.  The pidfile
> should
> > be private from random software anyways, other than the daemon itself and
> > perhaps whatever tools or init system are managing start/stop.
>
> Note that other programs might read your PID files too, for example
> systemd. And systemd won't take a lock on it (we don't want to make PID1
> deadlock...).


I guess whether systemd or another init system needs to read the PID file
is a design question, and my daemon design for this stuff tends to be
"different".  At least in my model, there's no reason for a normal init
system to need to read the pidfile - it's only used for exclusivity between
multiple instances of the same daemon to avoid races, and the fcntl() lock
information is used by status/stop/restart -like commands to identify a
running daemon without races or process misidentification.  Exactly one
daemon holds the fcntl writelock on the designated pidfile pathname, and
its PID is retrievable by reading the lock, which is released on daemon
shutdown or death.  And again, under systemd this can be made to work with
sd_notify of MAINPID.

Hence you need to make sure you write those files
> atomically anyway, and you cannot use locks for that, you have to use
> atomic renames... And if you do that, the locks become pretty useless...
>

Atomic renames (+ fsync!) get you a pidfile that's always readable by other
processes, but what they don't get you is solving the problem of races
between multiple instances of the daemon itself.  The data contained within
won't be reliable in a lot of real-world corner cases.  What happens when
someone issues a start command while another start command was still
executing?  Or a start while a stop was still executing, etc?  Almost every
scheme for pidfile management that tries to solve all the issues
surrounding it (guaranteeing a lack of two conflicting instances of the
same daemon, not losing track of a daemon by letting another instance
overwrite its pidfile while it's still running, "stop" commands never try
to mess with an unrelated process, and that status commands aren't confused
by them, etc) that I've ever seen has serious flaws.  I still think the
fcntl() method is the least-flawed of reasonably-portable methods.


> > IMHO, the method of taking an fcntl() writelock on a private pidfile
> > is one
>
> Also not that fnctl() (i.e. POSIX) locks are particularly broken
> actually, as they are removed whenever close() is invoked by your
> process on *any* fds to the file, even if you opened a completely
> independent fd. POSIX locks are excessively difficult to use properly,
> and in particular if you keep them over longer times...


Lots of things break if you, as the application developer, can't keep track
of where in your code you're messing a critical file like an
exclusively-locked pidfile.  Don't open other fds to the critical pidfile
owned by your daemonization code, problem solved?  Yes, POSIX locks are
difficult to use properly.  Almost everything about *nix daemonization is
difficult to do properly (isn't that one of the many things systemd is
trying to solve, at least for the Linux case?).


> > of the most reliable methods around for this stuff in the general case
> > outside of systemd-land.  AFAIK it's just about the only reliable way
> that
> > doesn't have inherent races or accidentally kill (or be DoS'd by)
> unrelated
> > processes in corner cases.  You don't even have to read the contents
> > of the
>
> Umm, no... POSIX locks are not NFS safe, and vulnerable, and really
> broken. They are a portability nightmare, and you claim to use them
> because of portability.... Also see this:
>
> http://0pointer.de/blog/projects/locking.html


Who puts daemon pidfiles on NFS (or at the very least, who would expect
them to grant lock-based daemon exclusivity across NFS)?  That's a whole
separate problem.  They're portable enough that they work for me on Linux
(systemd or not), *BSD, and MacOS.  I'm not trying to port to ancient
systems here.


> Anyway, this is not the place for discussing the brokeness of Linux file
> locking, so let's end this here...


IMHO, given that systemd replaces the init infrastructure for managing
long-running daemons, and given that it places a lot of practical and
conceptual limitations on what a daemon is and how it is managed versus a
traditional semi-portable *nixy way of doing things due to making very
large changes in this area, I think discussion of best practices for
portable daemons and how to best integrate those practices to accomplish
the same goals under systemd is *very* germane to this list.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/systemd-devel/attachments/20140519/8d3b6450/attachment.html>