[systemd-devel] [Lxc-users] Unable to run systemd in an LXC / cgroup container.

[Serge Hallyn]

Quoting Michael H. Warfield (mhw at WittsEnd.com):
> Sorry for taking a few days to get back on this.  I was delivering a
> guest lecture up at Fordham University last Tuesday so I was out of
> pocket a couple of days or I would have responded sooner...
> 
> On Mon, 2012-10-22 at 16:59 -0400, Michael H. Warfield wrote:
> > On Mon, 2012-10-22 at 22:50 +0200, Lennart Poettering wrote:
> > > On Mon, 22.10.12 11:48, Michael H. Warfield (mhw at WittsEnd.com) wrote:
> > > 
> > > > > > To summarize the problem...  The LXC startup binary sets up various
> > > > > > things for /dev and /dev/pts for the container to run properly and this
> > > > > > works perfectly fine for SystemV start-up scripts and/or Upstart.
> > > > > > Unfortunately, systemd has mounts of devtmpfs on /dev and devpts
> > > > > > on /dev/pts which then break things horribly.  This is because the
> > > > > > kernel currently lacks namespaces for devices and won't for some time to
> > > > > > come (in design).  When devtmpfs gets mounted over top of /dev in the
> > > > > > container, it then hijacks the hosts console tty and several other
> > > > > > devices which had been set up through bind mounts by LXC and should have
> > > > > > been LEFT ALONE.
> > > > 
> > > > > Please initialize a minimal tmpfs on /dev. systemd will then work fine.
> > > > 
> > > > My containers have a reasonable /dev that work with Upstart just fine
> > > > but they are not on tmpfs.  Is mounting tmpfs on /dev and recreating
> > > > that minimal /dev required?
> 
> > > Well, it can be any kind of mount really. Just needs to be a mount. And
> > > the idea is to use tmpfs for this.
> 
> > > What /dev are you currently using? It's probably not a good idea to
> > > reuse the hosts' /dev, since it contains so many device nodes that
> > > should not be accessible/visible to the container.
> 
> > Got it.  And that explains the problems we're seeing but also what I'm
> > seeing in some libvirt-lxc related pages, which is a separate and
> > distinct project in spite of the similarities in the name...
> 
> > http://wiki.1tux.org/wiki/Lxc/Installation#Additional_notes
> 
> > Unfortunately, in our case, merely getting a mount in there is a
> > complication in that it also has to be populated but, at least, we
> > understand the problem set now.
> 
> Ok...  Serge and I were corresponding on the lxc-users list and he had a
> suggestion that worked but I consider to be a bit of a sub-optimal
> workaround.  Ironically, it was to mount devtmpfs on /dev.  We don't

Oh, sorry - I take back that suggestion :)

Note that we have mount hooks, so templates could install a mount hook to
mount a tmpfs onto /dev and populate it.

Or, if everyone is going to need it, we could just add a 'lxc.populatedevs = 1'
option which does that without needing a hook.

devtmpfs should not be used in containers :)

-serge