[systemd-devel] Zero downtime restart of Web application (Nginx, Puma, JRuby)

[Kai Krakow]

Am Sat, 18 Jun 2016 13:56:03 +0200
schrieb Paul Menzel <paulepanter at users.sourceforge.net>:

> Dear systemd folks,
> 
> 
> the setup is as follows.
> 
> Nginx is used as the Web server, which communicates with a Ruby on
> Rails application over a socket. Puma [1] is used as the application
> server.
> 
> Nginx and Puma are managed by systemd service files.
> 
> If a new version of the application is installed, the goal is that
> there is no downtime. That means, until the application with the new
> code isn’t ready yet to serve requests, the old application still
> answers them, that means, not only are no requests lost, but during
> restart the visitor does not need to wait any longer than normally.
> 
> In this case, JRuby is used, which means that application start, `sudo
> systemctl start web-application.service` takes more than 30 seconds.
> 
> So, `sudo systemctl restart web-application.service` is not enough as
> Puma takes some time to be started. (The socket activation described
> in the Puma systemd documentation [2] only takes care, that no
> requests are lost. The visitor still has to wait.)
> 
> Is that possible by just using systemd, or is a load balancer like
> HAProxy or a special NGINX configuration and service file templates
> needed?
> 
> My current thinking would be, that a service file template for the
> Puma application server is used. The new code is then started in
> parallel, and if it’s done, it “takes over the socket”. (No idea if
> NGINX would need to be restarted.) Then the old version of the code
> is stopped. (I don’t know how to model that last step/dependency.)
> 
> What drawbacks does the above method have? Is it implementable?
> 
> How do you manage such a setup?

This is not really systemd specific. Systemd solves zero-downtime only
by socket-activation which is not exactly what you expect.

We are using mod_passenger with nginx to provider zero-downtime during
application deployment, background workers (job servers etc) are
managed by systemd and socket activation.

Passenger however does not support jruby afaik. But you may want to
look at how they implement zero downtime: Its kind of a proxy which is
switched to the new application instance as soon as it is up and
running. You could do something similar: Deploy to a new installation,
when ready, rewrite your nginx config to point to the new instance,
reload nginx, then gracefully stop the old instance. Its not that well
integrated as passenger does it but it can be optimized. However,
nothing of this is systemd specific except you still may want to use
socket activation in some places. Stopping and starting instances and
reloading nginx should be part of your deployment process. If controlled
with systemd, you can use service templates like
my-application at .service and then start
my-application at instance-201607150001.service and stop
my-application at instance-201607140003.service. You can use the instance
name to setup sockets, proxies etc.

-- 
Regards,
Kai

Replies to list-only preferred.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: Digitale Signatur von OpenPGP
URL: <https://lists.freedesktop.org/archives/systemd-devel/attachments/20160715/c553771a/attachment.sig>