[systemd-devel] systemd-nspawn isolation potentially causing issues with distccmon-text

[John]

On Wed, Nov 13, 2019 at 7:03 PM Uoti Urpala <uoti.urpala at pp1.inet.fi> wrote:
>
> On Wed, 2019-11-13 at 10:24 -0500, John wrote:
> > I am using systemd-nspawn to compile in a clean environment.  My
> > distcc cluster happily accepts requests from the container's build,
> > but the monitoring utility, distccmon-text, shows no output. I invoked
> > it defining the DISTCC_DIR variable to the correct directory in the
> > container.
>
> > Link to strace from the container:
> > https://gist.github.com/graysky2/0886025b60335de4c0b19ddf11f7aafb
>
> Your description is somewhat unclear. I'm assuming that this is
> actually a strace from OUTSIDE the container (as in, you are not
> running the distcc-mon program inside the container, but running it on
> the host system and only giving it a path to a filesystem location used
> by the in-container compilation process), and that this is the case you
> are trying to get working.

Correct.  I provided the 2nd strace (distcc outside and monitor
outside) as a positive control.

> I believe the problem is that the program reads PID values from the
> filesystem, but PIDs are not the same inside the container and outside.
> Thus recording a PID value inside the container and then trying to use
> that PID to find the same process from the host system will not work.
>
> If your container runs as a full enough machine with its own systemd
> and dbus, then the simplest solution is likely to run the monitoring
> utility in the container, for example with:
> machinectl shell <machine-name> <command>

The container is created by one of the Arch Linux build scripts
(https://git.archlinux.org/devtools.git/tree/arch-nspawn.in).  I do
not believe it runs with its own systemd/dbus.  When I am compiling
(https://git.archlinux.org/devtools.git/tree/makechrootpkg.in) in the
container, is there a way I can run distccmon-text from within the
same container to get it to read the same PID values?

I tried the following syntax but was met with a busy error:

# systemd-nspawn --directory /scratch/.chroot64/facade
DISTCC_DIR=/build/.distcc distccmon-text
Directory tree /scratch/.chroot64/facade is currently busy.

Appending the --ephemeral switch does not help:

# systemd-nspawn --directory /scratch/.chroot64/facade --ephemeral
DISTCC_DIR=/build/.distcc distccmon-text
Spawning container facade-8237d3321ae02886 on
/scratch/.chroot64/.#machine.facade197913f7d7ed5442.
Press ^] three times within 1s to kill container.
execv(DISTCC_DIR=/build/.distcc) failed: No such file or directory
Container facade-8237d3321ae02886 failed with error code 1.