[systemd-devel] Improving module loading

[Hoyer, Marko (ADITG/SW2)]

> -----Original Message-----
> From: Umut Tezduyar Lindskog [mailto:umut at tezduyar.com]
> Sent: Saturday, December 20, 2014 6:45 PM
> To: Hoyer, Marko (ADITG/SW2)
> Cc: systemd-devel at lists.freedesktop.org
> Subject: Re: [systemd-devel] Improving module loading
> 
> Hi Marko,
> 
> Thank you very much for your feedback!

You're welcome ;)

> 
> On Sat, Dec 20, 2014 at 11:45 AM, Hoyer, Marko (ADITG/SW2)
> <mhoyer at de.adit-jv.com> wrote:
> > Hi,
> >
> >> -----Original Message-----
> >> From: systemd-devel [mailto:systemd-devel-
> >> bounces at lists.freedesktop.org] On Behalf Of Umut Tezduyar Lindskog
> >> Sent: Tuesday, December 16, 2014 4:55 PM
> >> To: systemd-devel at lists.freedesktop.org
> >> Subject: [systemd-devel] Improving module loading
> >>
> >> Hi,
> >>
> >> Is there a reason why systemd-modules-load is loading modules
> >> sequentially? Few things can happen simultaneously like resolving
> the
> >> symbols etc. Seems like modules_mutex is common on module loads
> which
> >> gets locked up on few occasions throughout the execution of
> >> sys_init_module.
> >
> > We are actually doing this (in embedded systems which need to be up
> very fast with limited resources) and gained a lot. Mainly, IO and CPU
> can be better utilized by loading modules in parallel (one module is
> loaded while another one probes for hardware or is doing memory
> initialization).
> >
> >>
> >> The other thought is, what is the preferred way of loading modules
> >> when they are needed. Do they have to be loaded on ExecStartPre= or
> >> as a separate service which has ExecStart that uses kmod to load
> them?
> >> Wouldn't it be useful to have something like ExecStartModule=?
> >
> > I had such a discussion earlier with some of the systemd guys. My
> intention was to introduce an additional unit for module loading for
> exactly the reason you mentioned. The following (reasonable) outcome
> was:
> 
> Do you have links for the discussions, I cannot find them.

Actually not, sorry. The discussion was not done via any mailing list.

> systemd already has a service that loads the modules.

Sorry, there is a word missing in my sentence above. My idea was not to introduce a "unit" for modules loading but an own "unit type", such as .kmodule. The idea was to define .kmodule units to load one or a set of kernel modules each at a certain point during startup by just integrating them into the startup dependency tree. This idea would require integrating kind of worker threads into systemd. The outcome was as summarized below.

The advantages over systemd-modules-load are:
- modules can be loaded in parallel
- different sets of modules can be loaded at different points in time during startup

> 
> > - It is dangerous to load kernel modules from PID 1 since module
> > loading can get stuck
> > - Since modules are actually loaded with the thread that calls the
> > syscall, systemd would need additional threads
> > - Multi Threading is not really aimed in systemd for stability
> reasons
> > The probably safest way to do what you intended is to use an
> additional process to load your modules, which could be easily done by
> using ExecStartPre= in a service file. We are doing it exactly this way
> not with kmod but with a tool that loads modules in parallel.
> >
> > Btw: Be careful with synchronization. We found that lots of kernel
> modules are exporting device nodes in the background (alsa, some
> graphics driver, ...). With the proceeding mentioned above, you are
> moving the kernel module loading and the actual use of the driver
> interface very close together in time. This might lead to race
> conditions. It is even worse when you need to access sys attributes,
> which are exported by some drivers even after the device is already
> available and uevents have been sent out. For such modules, there
> actually is no other way for synchronization but waiting for the
> attributes to appear.
> 
> We are aware of the potential complications and races. But good to be
> reminded :)
> 

;) We actually stumbled over lots of things here while we rolled out this approach. Sometimes it is really funny that simple questions such as "What does your service actually need?" are hard to answer. It seems that sometimes things are working more or less accidently due to the fact that the udev trigger comes very early compared to the startup of the services.

> Umut
> 
> >
> >>
> >> Umut
> >> _______________________________________________
> >> systemd-devel mailing list
> >> systemd-devel at lists.freedesktop.org
> >> http://lists.freedesktop.org/mailman/listinfo/systemd-devel
> >
> > Best regards
> >
> > Marko Hoyer
> > Software Group II (ADITG/SW2)
> >
> > Tel. +49 5121 49 6948
> >


Best regards

Marko Hoyer
Software Group II (ADITG/SW2)

Tel. +49 5121 49 6948