[systemd-devel] How does hybrid cgroup setup work?

[Lennart Poettering]

On Fr, 10.11.17 11:11, Umut Tezduyar Lindskog (umut at tezduyar.com) wrote:

> On Wed, Nov 8, 2017 at 9:25 PM, Lennart Poettering
> <lennart at poettering.net> wrote:
> > On Sa, 28.10.17 22:01, Umut Tezduyar Lindskog (umut at tezduyar.com) wrote:
> >
> >> Hello,
> >>
> >> I am trying to have cpu controller on v1 and memory controller on
> >> unified but it is not working the way I am expecting. When I enable
> >> CONFIG_MEMCG, cgroup is popping up in /proc/cgroups and
> >> mount_cgroup_controllers() is mounting it very early. Once it is
> >> mounted in v1, it becomes unavailable in v2.
> >>
> >> Did I misunderstand how hybrid works?
> >
> > hybrid means that v2 is used only for tracking services (which is a
> > good thing, since it provides safe notification of when a cgroup
> > exits), but not for any of the controllers. That means hybrid mode is
> > mostly compatible with pure v1, except that there's yet another
> > hierarchy (the v2 one) and systemd uses it for its own purposes.
> 
> Is there a technical reason why we cannot have a broader hybrid like
> some controllers from v1 and some from v2?
> 
> We would like to keep using cpu v1 until cpu v2 is accepted but
> meanwhile take advantage of improvements on memory v2.

Well, right now, you have three types of setups:

1) legacy:

   /sys/fs/cgroup/ → tmpfs instance
   /sys/fs/cgroup/memory/ → memory controller hierarchy
   /sys/fs/cgroup/cpu/ → cpu controller heirarchy
   …
   /sys/fs/cgroup/systemd/ → named hierarchy, which systemd uses to
   manage its stuff (and which is mostly internal to systemd)

2) unified:

   /sys/fs/cgroup/ → the unified hierarchy (i.e, note that this is one
                     dir further up, in lieu of the tmpfs)

3) hybrid:

   /sys/fs/cgroup/ → tmpfs
   /sys/fs/cgroup/memory/ → memory controller hierarchy
   /sys/fs/cgroup/cpu/ → cpu controller heirarchy
   …
   /sys/fs/cgroup/systemd/ → named hierarchy, for compatibility
   /sys/fs/cgroup/unified/ → the unified hierarchy with no
   controllers, which systemd uses to manage its stuff (and which is
   mostly internal to systemd)

Now, supporting #1 and #2 definitely makes sense, as one is the old
setup, and one the future setup. #3 is in most ways compatible to #1,
as the only thing that changes was the addition of one more hierarchy,
but all the old controller hierarchies remain at the same place. Or in
other words, systemd's private hierarchy changes, but all the other
stuff remains at the exact same location

Now, what you propose, is neither similar to 1) or 3) (as the
controllers would be moved in part to the unified hierarchy). Nor
similar to #2 (as the the unified dir would not be /sys/fs/cgroup,
since then you'd have no place to mount the legacy controllers).

Hence, adding what you propose would only be a temporary stop-gap,
that at the moment of adding would already be clearly on its way out,
and I am very conservative of adding support now for something we know
will not survive for long...

And then there's also the big issue: the cgroup code is complex enough
given that we need to support three different setups. I'd really
prefer if we'd not add even more to that. In fact, I am really looking
forward for the day we can drop all cgroup support besides the unified
one from our tree. We could delete *so much* code then! And there's
only one thing hackers prefer over writing code: deleting code... ;-)

Lennart

-- 
Lennart Poettering, Red Hat