[systemd-devel] Ordering a service before remote-fs-pre.target makes it quite longer

[Andrei Borzenkov]

13.05.2020 21:57, Thomas HUMMEL пишет:
> Hello,
> 
> I'm using the xCAT (xcat.org) software to provision stateless HPC Centos
> 8.1 nodes. Via a systemd service called xcatpostinit1.service, it
> enables, at boot time, to run so called postscripts for, for instance
> 
> - configure eth nic with a manual (vs dhcp) NetworkManager profile
> - configure Infiniband nic
> - sync files from the xcat server.
> 
> I'm using it exactly for the 3 above examples. File syncing is quite
> light as it consists in syncing pre-created ssh hostkeys.
> 
> By default this service has got the following ordering dependency:
> 
> After=network.target rsyslog.service
> 

It does not match your graphs. Your service is apparently ordered after
network-online.target (not after network.target) and startup is most
certainly initiated before rsyslog.service. Not hat it explains anything
but at least you need to provide accurate facts when you ask question.

> and doesn't pull any dependency
> 
> For my own need, I added the following:
> 
> Before=beegfs-client.service
> Before=beegfs-helperd.service
> After=sshd-keygen.target
> 
> This works fine.
> 
> As one of the postscript this service runs adds a NetworkManager profile
> (to autoconnect from a dhcp-originating boot profile to a manual (same
> ip address) one, and since I nfs mount some filesystems I tought I
> should order units so as to setup network first and only then mount
> remote filesystems
> 
> I thus did add:
> 
> Before=remote-fs-pre.target
> 
> This has a funny result : it ultimately works but this
> xcatpostinit1.service then takes 1min+ to end vs 20sec when the latter
> dependency is not stated.
> 
> Please find here to systemd-analyze plot svg's reflecting the with
> Before=remote-fs-pre.target (boot-dep.svg) and without it (boot-nodep.svg)
> 
> http://dl.pasteur.fr/fop/GCPbmpii/boot-dep.svg
> 
> http://dl.pasteur.fr/fop/AcfI7CSh/boot-nodep.svg
> 
> I did spend a lot of time trying to figure out why such a difference,
> all things being equal otherwise.
> 
> The part of the service which takes time is the syncing of files which
> 
> - occurs before the network reconfiguring
> - consists in rsync'ing files from server to node
> - triggered by a REST API call (http/80) for what I saw in sources
> 
> I did not see any cycles nor anything that caught my eye turning systemd
> in debug mode neither.
> 
> As remote-fs-pre.target is a special target I thought I may did misuse
> it for that matter.
> 
> Can you help me figure out why the difference ?
> 


If startup of your service was initiated (as opposed to delaying
startup) it is really outside of systemd scope. Systemd has no control
over what your service does once ExecStart is spawned. You need to debug
your service to find out what happens.