[systemd-devel] systemd leaving empty cgroups when exiting a container?

[Lars Kellogg-Stedman]

I think this another intersection-of-systemd-and-lxc question...

If I stop a container using 'lxc-stop', subsequent attempts to start
that containr will result in the following error:

  # lxc-start -n node0
  lxc-start: Device or resource busy - failed to remove previous cgroup '/sys/fs/cgroup/systemd/node0'
  lxc-start: failed to spawn 'node0'
  lxc-start: Device or resource busy - failed to remove cgroup '/sys/fs/cgroup/systemd/node0'

There does indeed exist a cgroup by this name:

  # find /sys/fs/cgroup -name node0
  /sys/fs/cgroup/systemd/node0

But it has no tasks:

  # cat /sys/fs/cgroup/systemd/node0/tasks
  #

It does, however, have a number of child cgroups, which is why it can't
be removed.  I can remove it manually with find and xargs, but I'm
trying to figure out how to avoid the situation from cropping up in the
first place.

What's interesting is that the problem does *not* occur if I stop the
container by running "halt" inside the container...so I'm guessing that
using lxc-stop means that something (systemd inside the container?)
doesn't get the chance to clean up properly.

A common suggestion is to set up a release_agent to remove cgroups when
they are empty, but this hierarchy already has a release agent:

  # cat /sys/fs/cgroup/systemd/release_agent
  /usr/lib/systemd/systemd-cgroups-agent

It looks like this sends some sort of notification to systemd via dbus.

So...

Is there a way to get containers cleaned up properly even when using
lxc-stop? Or is there a way to get systemd to remove cgroups as they
become empty?  Or can I replace the systemd release agent with my own?

Thanks for your suggestions,

-- 
Lars Kellogg-Stedman <lars at oddbit.com>