[systemd-devel] sometimes systemd takes a very long time to restart service

[JB]

Andrey Borzenkov wrote:
> В Tue, 01 Jan 2013 23:37:56 -0700
> JB <general at itpsg.com> пишет:
>   
>> Andrey Borzenkov wrote:
>>     
>>> В Tue, 01 Jan 2013 18:52:38 -0700
>>> JB <general at itpsg.com> пишет:
>>>       
>>>> Thanks!  I'll try and it may work in my case.  What's interesting is 
>>>> that in your case it sounded like rsyslog was hanging around while it 
>>>> was having problems dealing with the condition of having the network 
>>>> unavailable.  In my case, webrickd actually stops and shuts down almost 
>>>> immediately but for some reason systemd doesn't or can't figure that 
>>>> out.
>>>>         
>>> Showing "systemctl status webrickd.service" before restarting and
>>> during restart may give some hints.
>>>       
>> Good idea.
>>
>> It goes from active/running to this immediately after restart...
>>
>> webrickd.service - Configuration ruby webrick daemon
>>           Loaded: loaded (/etc/systemd/system/webrickd.service)
>>           Active: deactivating (final-sigterm) since Tue, 01 Jan 2013 
>> 22:44:39 -0700; 1min 14s ago
>>          Process: 15771 ExecStart=/home/rtuser/app/bin/webrickd.rb -d -p 
>> /home/rtuser/app/data/logs/webrickd.pid (code=exited, status=0/SUCCESS)
>>
>> Then this, anywhere from 1 to 3 minutes later...
>>
>> webrickd.service - Configuration ruby webrick daemon
>>           Loaded: loaded (/etc/systemd/system/webrickd.service)
>>           Active: active (running) since Tue, 01 Jan 2013 22:47:40 
>> -0700; 15s ago
>>          Process: 15807 ExecStart=/home/rtuser/app/bin/webrickd.rb -d -p 
>> /home/rtuser/app/data/logs/webrickd.pid (code=exited, status=0/SUCCESS)
>>     
>
> And initial state (i.e. after it is started)?
>
>   
>> Any ideas why it sits in state "deactivating (final-sigterm)" for 1 to 3 
>> minutes
It gets better.  I took a very stripped simple daemon to try to 
understand the interaction between systemd and the process.  Here it is:
*******************************************
#!/bin/env 
/usr/bin/ruby                                                                                                                    

require "webrick"

class SomeServlet < WEBrick::HTTPServlet::AbstractServlet
  def do_GET (request, response)
    response.status=200
    response.body="Howdy"
  end
end

class SomeServer < WEBrick::HTTPServer
  def initialize
    super(:Port => 1234, :DocumentRoot => ".")
    mount "/", SomeServlet
    ['TERM'].each { |signal| trap(signal) { shutdown } 
}                                                                                   

    start
  end
end

WEBrick::Daemon.start
s = SomeServer.new
*******************************************


Here's the service file:
*******************************************
[Unit]
Description=Webrick Test Service
After=network.target

[Service]
Type=forking
ExecStart=/usr/bin/ruby /home/rtuser/test.rb

[Install]
WantedBy=multi-user.target
*******************************************

I put this in /etc/systemd/system/webrickd.service
Then ran systemctl enable webrickd.service
Then ran systemctl --system daemon-reload
Then ran systemctl start webrickd.service

Running it from the command line runs as it should.  Web server starts 
and test.rb is running in the background as a daemon returning me back 
to the shell.

But as soon as I try to use systemd, it starts but as soon as the 
WEBrick::Daemon.start call is made, everything is killed.
Nothing happens after that.  I've tried using the "simple" service 
type.  I've tried trapping different signals, I've tried starting it up 
different ways (e.g. just /home/rtuser/test.rb instead of call to ruby 
first.)  I've tried just starting the server and not calling 
WEBrick.Daemonize then systemd reports all kinds of errors and 
failures.  I've tried all kinds of things.  I opened a log file both 
before and after the WEBrick::Daemon.start call to see what was and was 
not getting executed.  Before call to WEBrick::Daemon.start everything 
works, afterward everything is dead but the systemctl command exits quickly.

systemctl status shows:
webrickd.service - Webrick Test Service
          Loaded: loaded (/etc/systemd/system/webrickd.service)
          Active: inactive (dead) since Wed, 02 Jan 2013 01:23:01 -0700; 
9min ago
         Process: 1605 ExecStart=/usr/bin/ruby /home/rtuser/test.rb 
(code=exited, status=0/SUCCESS)
        Main PID: 1607 (code=exited, status=0/SUCCESS)
          CGroup: name=systemd:/system/webrickd.service

I'm really trying to understand systemd but it is times like this when 
do long for the simpler days of init.  At this point I would *gladly* 
trade a longer boot time for all the headache this has given me.  I had 
a working init script that has run great for years but quit working with 
systemd so I tried the systemd way and just can't make this stupid thing 
work.  It either takes forever or it flat out fails.

If someone could tell me the "right way" to make this work with systemd, 
I would love to use it but I've been at this on and off for weeks and it 
isn't getting any easier.  From my perspective systemd appears rigid and 
quite unforgiving.  I can't call it buggy yet because I don't know that 
I've found any but it sure doesn't seem nearly as simple and easy to get 
something running at startup as the documentation would have me believe.