<div dir="ltr">On Wed, Feb 24, 2016 at 4:29 AM, Lennart Poettering <span dir="ltr"><<a href="mailto:lennart@poettering.net" target="_blank">lennart@poettering.net</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Tue, 23.02.16 14:58, Ben Woodard (<a href="mailto:woodard@redhat.com">woodard@redhat.com</a>) wrote:<br> <br> > On Thu, Feb 18, 2016 at 9:08 AM, Lennart Poettering <<a href="mailto:lennart@poettering.net">lennart@poettering.net</a>><br> > wrote:<br> ><br> > > On Wed, 17.02.16 17:44, Ben Woodard (<a href="mailto:woodard@redhat.com">woodard@redhat.com</a>) wrote:<br> > ><br> > > > Is it intentional that systemd holds a reference to a socket it has<br> > > > just accepted even though it just handed the open socket over to a<br> > > > socket.activated service that it has just started.<br> > ><br> > > Yes, we do that currently. While there's currently no strict reason to<br> > > do this, I think we should continue to do so, as there was always the<br> > > plan to provide a bus interface so that services can rerequest their<br> > > activation and fdstore fds at any time. Also, for the listening socket<br> > > case (i.e. Accept=no case) we do the same and have to, hence it kinda<br> > > is makes sure we expose the same behaviour here on all kinds of<br> > > sockets.<br> > ><br> ><br> > I'm having trouble believing that consistency of behavior is an ideal to<br> > strive for in this case. Consistency with xinetd's behavior would seem to<br> > be a better benchmark in this case.<br> <br> </span>Well, we are implementing our own socket passing protocol anyway, and<br> support the inetd style one only for completeness.<br></blockquote><div><br></div><div>Which I believe supports my assertion that for the legacy inetd style socket activation, we reproduce its semantics so that people porting systemd have a measure of backwards compatibility. Then for those services which are updated to the new socket passing protocol they can have whatever semantics are needed to support that new kind of interaction.</div><div><br></div><div>It should be pretty obvious from the unit file when the service is expecting simple inetd semantics vs implementing the socket passing interface. </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <span class=""><br> > And I'm not sure that I understand the value being able to rerequest a<br> > fdstore of fds. To me this sounds like it would be a very rarely used<br> > feature. Could this be made an option that could be enabled when you add<br> > the bus service that allows a service to rerequest their activation and<br> > fdstore fds?<br> <br> </span>Apple's launchd does activation that way. They have a "check-in"<br> protocol, where instead of passing fds through exec() the activated<br> process ask for them via an IPC call. This has quite some benefits as<br> it allows the service manager to update the set of sockets<br> dynamically, and the service can query them at any time to acquire the<br> newest set.<br> <br></blockquote><div><br></div><div>I think it is good that you are patterning this off of apple's prior art. However, I'm still moderately skeptical about the utility of this. It is interesting; it sounds cool; I have no idea when I would ever want to use it and can't think of any cases where a service that I'm familiar with needs that feature. I'm not in anyway trying to lobby against it. I just feel like these future plans shouldn't prevent a slightly more complete backwards compatibility mode for inetd socket activation services.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Another reason btw to keep the fd open in systemd is that we allow a<br> series of programs to be invoked via multiple ExecStartPre=,<br> ExecStart=, ExecStartPost=, ExecStop=, ExecStopPost= lines, and we<br> have to keep things open at least for all but the last one hence, so<br> that we have something to pass to the last invocation.<br></blockquote><div><br></div><div>I think that is the most convincing argument thus far. I'd still argue for special case logic that with simple socket activation accept=yes when the last invocation is made, systemd closes its reference to the socket.</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <span class=""><br> > > Did you run into problems with this behaviour?<br> > ><br> ><br> > Oh yes. It breaks a large number of management tools that we have on to do<br> > various things on clusters. It is a kind of pseudoconcurrency. Think of<br> > like this:<br> ><br> > foreach compute-node;<br> > rsh node daemonize quick-but-not-instant-task<br> ><br> > With xinetd the demonization would close the accepted socket. Then the<br> > foreach loop would nearly instantly move onto the next node. We could zip<br> > through 8000 nodes in a couple of seconds.<br> ><br> > With systemd holding onto the socket the "rsh" hangs until the<br> > quick-but-not-instant-task completes. This causes the foreach loop to take<br> > anywhere from between 45min and several hours. Because it isn't really rsh<br> > and demonize and I just used that to make it easy to understand what is<br> > going on, rewriting several of our tools is non-trivial and would end up<br> > violating all sorts of implicit logical layering within the tools and<br> > libraries that we use to build them.<br> <br> </span>Hmm, let me get this right: the activated service closes its fd much<br> earlier than it calls exit() and you want that this is propagated back<br> to the client instead of having that delayed until the service<br> actually calls exit()?<br></blockquote><div><br></div><div>sort of but not quite.</div><div>More like this:</div><div><br></div></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div>systemd</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div> accept</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div> ...</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div> fork</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div> if child</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div> ExecStart service-process</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div>service-process</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div> # this is what I mean by daemonize </div></div></div><div class="gmail_extra"><div class="gmail_quote"><div> fork</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div> if parent</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div> exit</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div> close(stdin)</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div> close(stdout)</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div> close(stderr) # at this point only systemd has a reference to the socket that was accepted</div></div></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div> # the expectation here is the socket will be shutdown </div><div> # and the client which initiated the connection can move on</div></div></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div><br></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div> do-something # something that takes a few seconds</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div> exit # this is when systemd closes the socket and the client on the other end can move on</div></div></div></blockquote><div class="gmail_extra"><div class="gmail_quote"><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <br> Normally one would solve this by inserting shutdown(fd, SHUT_RDWR) at<br> the right place, since that *really* terminates the connection,<br> regardless if anyone else has an fd open still. Is that an option<br> here?<br></blockquote><div><br></div><div>I didn't actually try that myself but that was the 2nd thing that I suggested when I got the original problem report and the developer who reported the problem told me that it didn't work. </div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <br> I figure I'd be open to adding an option that makes sure the<br> connection socket is closed as soon as the last ExecXYZ= process we<br> need is forked off.<br> <span class=""><br> > Where is this in the source code? I've been planning to send you a patch to<br> > change the behavior but I have't quite connected the dots from where a job<br> > within a transaction is inserted on the run queue to where the fork for the<br> > "ExecStart" actually happens.<br></span></blockquote><div><br></div><div>That would be great but I still think that it should be the default when it is a very simple xinetd style socket activated service. Advanced and fancy are capable and flexible when you need them. However, allowing the very simple is important too.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=""> <br> </span>it's in src/core/service.<br> <div class="HOEnZb"><div class="h5"><br> Lennart<br> <br> --<br> Lennart Poettering, Red Hat<br> </div></div></blockquote></div><br></div></div>