[systemd-devel] mysqld about back to use mysqld_safe in Fedora

[Lennart Poettering]

On Wed, 27.06.12 20:21, Tom Lane (tgl at redhat.com) wrote:

> Well, it may be a hack rather than the nicest possible solution, but
> here's the thing: every production installation of mysql in the world
> runs underneath mysqld_safe, and has done for the last ten years or
> more.  The server's behavior has been designed, or at least evolved, to
> fit that script's behavior and vice versa.  If we try to use systemd's
> restart functionality with a slightly different set of conditions for
> restarting after crashes, we are likely to be finding ourselves
> endlessly patching the server to make its corner-case behaviors fit
> systemd.  Patches which there is no chance upstream will take back,
> because they'll say "why don't you use mysqld_safe like everybody else
> does?"

Well, this is Free Software. Patches speak louder than words, and most
people I noticed are actually quite interested in good, working, correct
code. Before giving up, please always try to fix upstream, before
working around something in downstream.

> So basically, what I foresee here is unproductive make-work, and so
> I don't want to go down that path.  Getting rid of one usually-idle
> shell process isn't valuable enough to justify ongoing maintenance
> headaches.

Well, it's not just about that. It's that the whole problem reporting
logic is built on that. i.e. we'll generate journal events when services
fail, and things like that. All that is lost if you catch all errors in
shell, and try to work on that on your own.

Sooner or later server management consoles wil hook into the failure
logic of systemd, and will see all problems, but MySQL's. Also, the HA
folks are interested into syncing their heartbeat logic into systemd's
failure/restart logic. All that won't work if MySQL wants to do its own
thing here.

> >> Another issue is the PID file guessing.
> 
> > Hmm, does mysql fork on its own? 
> 
> mysqld proper does not.  The reason we have to use type=forking is
> that we're launching mysqld_safe and having it prepare the arguments for
> mysqld (which is quite a few hundred lines of shell code we'd rather not
> reimplement), then launch the daemon, and then exit (instead of waiting
> for the daemon to terminate as it usually does).  So from systemd's
> perspective it's a forking server and PID guessing (or a PIDfile) is
> necessary.

Hmm, I am not following, I thought this thread was about bypassing mysqld_safe?

> We could dodge that problem by having the script "exec" the daemon
> instead of launching it as a subprocess.  If memory serves, we tried
> that initially and it didn't work because SELinux objected.  We could
> of course get that detail of the security policy changed, and probably
> would if we were going to stick with this path.

SELinux policy is changed all the time. Dan Walsh is fantastic at
correcting the policy in amazingly short turnaround times!

> > Normally a SysV daemon should have a clear initialization phase that it
> > informs the init system about by double-forking (and returning in the
> > parent). And the PID file should be written in that initialization
> > phase, and the parent should return *only* after the PID file has been
> > sucessfully written.
> 
> Unfortunately, you're just spinning theories out of whole cloth here
> about what a daemon "should" do.  I don't think reality agrees with
> this.  

In general this is how things work, but there are of course exeptions
where this is not working as it should, MySQL apparently being one of them.

> Certainly neither mysql nor postgresql have ever guaranteed any
> such thing.  Historically their init scripts have waited until the
> server responds to client contacts before supposing the service is up,
> anyway --- the existence of a pidfile just isn't that interesting from
> the perspective of a client wanting to know whether the service is up,
> or even whether it can be expected to come up shortly.  (In both
> databases, there can be a long and not-certain-to-succeed data recovery
> operation occurring after the pidfile is established.  It's typical to
> not want to consider the service "up" till that's complete.)

Such busy loops of sleeping, trying, sleeping, retrying, ... are really
not how we want to do things these days.
> 
> > I think the best way to solve this really is to fix MySQL to either only
> > exit in the parent when initialization is complete, or to invoke
> > sd_notify() as described above.
> 
> Basically, we have no interest in maintaining such patches forever,
> and no expectation that upstream would take them back either.  If that
> aspect were the only rough spot in using systemd we might be willing
> to trouble over it, but there are enough other showstoppers in front
> of it that we're just not going to reach that question.

I think most healthy projects follow the policy of "upstream
everything", and so should we here.

Lennart

-- 
Lennart Poettering - Red Hat, Inc.