[systemd-devel] [Lxc-users] Unable to run systemd in an LXC / cgroup container.

[Michael H. Warfield]

On Thu, 2012-10-25 at 11:19 -0500, Serge Hallyn wrote:
> Quoting Michael H. Warfield (mhw at WittsEnd.com):
> > Sorry for taking a few days to get back on this.  I was delivering a
> > guest lecture up at Fordham University last Tuesday so I was out of
> > pocket a couple of days or I would have responded sooner...
> > 
> > On Mon, 2012-10-22 at 16:59 -0400, Michael H. Warfield wrote:
> > > On Mon, 2012-10-22 at 22:50 +0200, Lennart Poettering wrote:
> > > > On Mon, 22.10.12 11:48, Michael H. Warfield (mhw at WittsEnd.com) wrote:
> > > > 
> > > > > > > To summarize the problem...  The LXC startup binary sets up various
> > > > > > > things for /dev and /dev/pts for the container to run properly and this
> > > > > > > works perfectly fine for SystemV start-up scripts and/or Upstart.
> > > > > > > Unfortunately, systemd has mounts of devtmpfs on /dev and devpts
> > > > > > > on /dev/pts which then break things horribly.  This is because the
> > > > > > > kernel currently lacks namespaces for devices and won't for some time to
> > > > > > > come (in design).  When devtmpfs gets mounted over top of /dev in the
> > > > > > > container, it then hijacks the hosts console tty and several other
> > > > > > > devices which had been set up through bind mounts by LXC and should have
> > > > > > > been LEFT ALONE.
> > > > > 
> > > > > > Please initialize a minimal tmpfs on /dev. systemd will then work fine.
> > > > > 
> > > > > My containers have a reasonable /dev that work with Upstart just fine
> > > > > but they are not on tmpfs.  Is mounting tmpfs on /dev and recreating
> > > > > that minimal /dev required?
> > 
> > > > Well, it can be any kind of mount really. Just needs to be a mount. And
> > > > the idea is to use tmpfs for this.
> > 
> > > > What /dev are you currently using? It's probably not a good idea to
> > > > reuse the hosts' /dev, since it contains so many device nodes that
> > > > should not be accessible/visible to the container.
> > 
> > > Got it.  And that explains the problems we're seeing but also what I'm
> > > seeing in some libvirt-lxc related pages, which is a separate and
> > > distinct project in spite of the similarities in the name...
> > 
> > > http://wiki.1tux.org/wiki/Lxc/Installation#Additional_notes
> > 
> > > Unfortunately, in our case, merely getting a mount in there is a
> > > complication in that it also has to be populated but, at least, we
> > > understand the problem set now.
> > 
> > Ok...  Serge and I were corresponding on the lxc-users list and he had a
> > suggestion that worked but I consider to be a bit of a sub-optimal
> > workaround.  Ironically, it was to mount devtmpfs on /dev.  We don't

> Oh, sorry - I take back that suggestion :)

Well, it worked (sort of) and reinforced what the problem was and where
the solution lay so there's no need to be sorry for it.  We learned and
we know why it's not the right solution.  This is good.  We made a lot
of progress on this just in the last week.  This is very good.

> Note that we have mount hooks, so templates could install a mount hook to
> mount a tmpfs onto /dev and populate it.

Ah, now that is interesting.  I haven't looked at that before.  I need
to explore that further.

> Or, if everyone is going to need it, we could just add a 'lxc.populatedevs = 1'
> option which does that without needing a hook.

Eventually, with Fedora (and later RHEL / CentOS / SL), Arch Linux, and
others going to systemd, I think this is going to be needed sooner than
later.

> devtmpfs should not be used in containers :)

Concur!

> -serge

Regards,
Mike
-- 
Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw at WittsEnd.com
   /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9          | An optimist believes we live in the best of all
 PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
URL: <http://lists.freedesktop.org/archives/systemd-devel/attachments/20121025/dbdb3154/attachment-0001.pgp>