[systemd-devel] systemd and cgroup memory membership reset

[Lennart Poettering]

On Tue, 17.05.16 21:50, Tomasz Janowski (tomjanowski at gmail.com) wrote:

> Hello,
> 
>  I have an issue with possible conflicts between systemd and a SLURM
> scheduler cgroup management. SLURM ca be configured to use a few cgroup
> subsystems like freezer, cpuset and memory. The daemon itself runs in the
> root of the memory tree, but it creates entries (subdirectories) for child
> processes to constrain their memory use.
> 
> Occasionally it happens that processes started by SLURM get their memory
> membership reset (e.g. after a week of running). The cgroup tree is there,
> only processes are relocated back to the root of memory cgroup. Is there
> any action that a system administrator can take, or a cron script that can
> reset memory cgroup membership while keeping the other two intact (freezer,
> cpuset)? This would affect only child processes, as a master (slurmd) runs
> in the root of memory cgroup.
> 
> I use systemd 215 and the options relevant to memory are:
> 
> MemoryAccounting=no
> MemoryLimit=18446744073709551615
> 
> I am quite sure that this happens due to systemd actions. Any remarks would
> be greatly appreciated! Otherwise I will have to plow through the sources
> trying to figure this out. :(

Well, it's simply not supported to have two cgroup managers manage the
same cgroup hierarchy. This must break as the cgroup logic is really
not designed to allow cooperative management. In fact, the kernel guys
are pretty explicit about cgroups being a single-writer interface.

Hence, your cgroup software is not compatible with systemd. If it
shall run in conjunction with systemd, then it may not intefere with
the top-level cgroup hierarchy at all.

That all said, systemd permits delegation of subhierarchies, via the
Delegate= property in service and unit files. If that's set for a
unit, then systemd won't migrate processes below that subhierarchy,
and it may be managed by another manger. This is how container
managers can have their own subtrees for example.

Also see:

https://www.freedesktop.org/wiki/Software/systemd/ControlGroupInterface/

Note that the 215 release you are using is really ancient however,
hence, YMMV may vary with all of this.

Lennart

-- 
Lennart Poettering, Red Hat