Help building Flatpaks in an unpriviliged Docker container

[Dan Nicholson]

On Tue, Dec 3, 2019 at 4:43 AM Nimrod Adar <nimrod at endlessm.com> wrote:
>
> Hi,
>
> I'm trying to build the Endless SDK inside an unprivileged Docker container, but unsuccessfully. The failure is running Bubblewrap inside the container (bwrap: capset failed: Operation not permitted). All of the files (Dockerfile, manifest, etc.) and the steps I ran to build the container image and the SDK are in https://github.com/adarnimrod/flatpak-docker-example. I tried running with the seccomp policy from https://github.com/flatpak/flatpak-docker-images/blob/master/flatpak-seccomp.json, with seccomp and apparmor disabled, with user namespace enabled, using both root and a regular user inside the container. The only way I could successfully build the SDK inside a Docker container was using a privileged container. Any help would be greatly apprec iated.

Where in the build is it failing? Can you show a log or just the end?

Any chance you can run under strace to see how capset is being called?
There are a couple places where capset is called in bwrap and I'd like
to know what's causing the EPERM. See also
https://github.com/containers/bubblewrap/pull/122, which has to do
with running bubblewrap under systemd-nspawn. That would be a similar
situation to docker, however that issue is about seccomp filtering
capset, which you've worked around here. I also believe that if you
fail in seccomp land then your process gets killed with SIGSYS, which
wouldn't get you the EPERM "Operation not permitted message".

My hunch is that the capability bounding set that docker uses is
tighter than what bwrap is trying to use. Can you enter the container
and run "capsh --print" to see what the capabilities are? You can
compare that to what the capability list is under --privileged and try
some different things. Actually, looking at bwrap, here's the list of
capabilities it wants to set:

CAP_SYS_ADMIN
CAP_SYS_CHROOT
CAP_NET_ADMIN
CAP_SETUID
CAP_SETGID
CAP_SYS_PTRACE

>From the docker I have, CAP_SYS_ADMIN, CAP_NET_ADMIN and
CAP_SYS_PTRACE are missing from the set. So, I'd try running docker
with --cap-add SYS_ADMIN --cap-add NET_ADMIN --cap-add SYS_PTRACE and
seeing if that works.

It would also be interesting to know if bwrap is being called with
either of the --cap-add or --cap-drop options since it obviously has
to use capset in that situation. Looking at flatpak I believe it only
uses --cap-drop when installing extra data, which I don't think should
happen when building our runtime, but I'm not totally sure about that.

Also, what Docker image are you using for the build? I'm curious if
bwrap is installed setuid.