[systemd-devel] Unable to run systemd in an LXC / cgroup container.

[Kay Sievers]

On Sun, Oct 21, 2012 at 11:25 PM, Michael H. Warfield <mhw at wittsend.com> wrote:
> This is being directed to the systemd-devel community but I'm cc'ing the
> lxc-users community and the Fedora community on this for their input as
> well.  I know it's not always good to cross post between multiple lists
> but this is of interest to all three communities who may have valuable
> input.
>
> I'm new to this particular list, just having joined after tracking a
> problem down to some systemd internals...
>
> Several people over the last year or two on the lxc-users list have been
> discussions trying to run certain distros (notably Fedora 16 and above,
> recent Arch Linux and possibly others) in LXC containers, virualizing
> entire servers this way.  This is very similar to Virtuoso / OpenVZ only
> it's using the native Linux cgroups for the containers (primary reason I
> dumped OpenVZ was to avoid their custom patched kernels).  These recent
> distros have switched to systemd for the main init process and this has
> proven to be disastrous for those of us using LXC and trying to install
> or update our containers.
>
> To put it bluntly, it doesn't work and causes all sorts of problems on
> the host.
>
> To summarize the problem...  The LXC startup binary sets up various
> things for /dev and /dev/pts for the container to run properly and this
> works perfectly fine for SystemV start-up scripts and/or Upstart.
> Unfortunately, systemd has mounts of devtmpfs on /dev and devpts
> on /dev/pts which then break things horribly.  This is because the
> kernel currently lacks namespaces for devices and won't for some time to
> come (in design).  When devtmpfs gets mounted over top of /dev in the
> container, it then hijacks the hosts console tty and several other
> devices which had been set up through bind mounts by LXC and should have
> been LEFT ALONE.
>
> Yes!  I recognize that this problem with devtmpfs and lack of namespaces
> is a potential security problem anyways that could (and does) cause
> serious container-to-host problems.  We're just not going to get that
> fixed right away in the linux cgroups and namespaces.
>
> How do we work around this problem in systemd where it has hard coded
> mounts in the binary that we can't override or configure?  Or is it
> there and I'm just missing it trying to examine the sources?  That's how
> I found where the problem lay.

As a first step, this probably explains most of it:
  http://www.freedesktop.org/wiki/Software/systemd/ContainerInterface

Kay