[systemd-devel] nspawn: devpts not mounted with PrivateUsers

[Lennart Poettering]

On Wed, 19.04.17 12:03, Olaf the Lost Viking (olaf.the.lost.viking at gmail.com) wrote:

> == Unexpected behaviour:
> Setting up and running nspawn based containers without any PrivateUsers-
> setting works. The containers run using a random user-id. (Here I seem to 
> misunderstand the manual as it says "false" is the default setting and 
> therefore no mapping at all should happen.)
> 
> But as soon as I add a PrivateUsers=true or PrivateUsers=ID setting into the 
> corresponding .nspawn-file, systemd fails while mounting devpts in the 
> container.

What does "fail" mean? Do you get a message about it? Or do you simply
miss it if you look at /dev/pts inside the container?

> 
> Since I like the idea of having a dedicated user for each container (and 
> therefore seeing his uid in ps & co on the host), I did the following:
> 
> == On the host:
> $ groupadd -g3777036288 MY_GROUP

Don't do this. If you register the group like this, nspawn will
normally abstain from using this group. Use "nss-mymachines" instead
(consider lobbying your distro to turn it on automatically when
nspawn/machined are installed) which will make all private UIDs used
by nspawn (or any other machined registered containers) appear in the
user database (as shown by getent passwd) without any modification of
/etc/passwd or any other file therein.

> $ echo MY_GROUP:3777036288:65536 >> /etc/subgid

"subuid" and "subgid" is not used by nspawn. Quite frankly, I am
pretty sure these files are pretty broken concepts...

> $ useradd -d/var/lib/machines/MY_CONTAINER -M -g3777036288 -u3777036288 MY_USER
> $ echo MY_USER:3777036288:65536 >> /etc/subguid
> $ chown MY_USER:MY_GROUP /var/lib/machines/MY_CONTAINER
> $ echo -e "[Exec]\nPrivateUsers=true\n" > /etc/systemd/nspawn/MY_CONTAINER.nspawn
> -OR-
> $ echo -e "[Exec]\nPrivateUsers=3777036288\n" > /etc/systemd/nspawn/MY_CONTAINER.nspawn
> $ machinectl start MY_CONTAINER
> 
> (The strangely looking ID represents the container in the upper 16 bits so 
> that nspawn can use the lower 16 bits for the local uids. And not putting 
> anything in /etc/sub?id doesn't change anything. But putting the IDs there is 
> the correct way, right?)

As suggested above: don't make any changes in /etc all (just make sure
nss-mymachines) is enabled.
> 
> Journalctl shows the following:
> 
> == On the host:
>   systemd[1]: Starting Container MY_CONTAINER...
>   systemd-nspawn[6345]: Selected user namespace base 3777036288 and range 65536.
>   systemd-nspawn[6345]: Failed to mount n/a on /var/lib/machines/MY_CONTAINER/sys/fs/selinux (MS_BIND ""): No such file or directory
>   systemd-nspawn[6345]: Failed to mount n/a on
>   /var/lib/machines/MY_CONTAINER/sys/fs/selinux
>   (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_REMOUNT|MS_BIND ""):
>   Invalid argument

I figure we should downgrade these two messages when selinux is not available.

>   systemd-nspawn[6345]: Failed to mount devpts on
>   /var/lib/machines/MY_CONTAINER/dev/pts (MS_NOSUID|MS_NOEXEC
>   "newinstance,ptmxmode=0666,mode=620,gid=3777036293"): Invalid
>   argument

Because Linux is really really broken, devpts' gid= parameter is
parsed as signed integer, and then cast to an unsigned one. This
effectively prohibits using any UIDs > 2^31-1 for containers, if you
want to use devpts... If you try anyway, you get EINVAL back... In
other words, pick a lower UID base (or let nspawn pick it for you),
and it should work.

Somebody should really fix that in the kernel, but I fear type safety
and stuff aren't well understood in the kernel community...

Lennart

-- 
Lennart Poettering, Red Hat