[systemd-devel] File system gets remounted read-only after using nspawn

[Canek Peláez Valdés]

(Sorry for taking so long to answer, I was away from my test system).

On Sat, Dec 7, 2013 at 5:56 PM, Colin Guthrie <gmane at colin.guthr.ie> wrote:
> 'Twas brillig, and Canek Peláez Valdés at 05/12/13 23:26 did gyre and
> gimble:
>> Hi; I've been playing with systemd-nspawn with Gentoo. I'm running
>> systemd 208, and the normal Gentoo stage3 image [1], installing
>> systemd 204 inside it. Everything works great, but for some reason
>> after powering off the container, the file system where it resides
>> gets remounted read-only. I should notice that this is a file system
>> different from /.
>
> Interesting. Can't say I've seen this personally. Only thing I've seen
> that is even semi related is that when playing with containers recently,
> the service inside the container that remounts root filesystem rw
> failed. Not seem much else other than that tho'.

I don't think that's the problem; I don't see any error message when
starting or shutting down the container:

centurion ~ # systemd-nspawn -bD /files/gentoo
Spawning namespace container on /files/gentoo (console is /dev/pts/9).
Init process in the container running as PID 2841.
systemd 204 running in system mode. (+PAM +LIBWRAP -AUDIT -SELINUX
+IMA -SYSVINIT -LIBCRYPTSETUP -GCRYPT +ACL -XZ)
Detected virtualization 'systemd-nspawn'.

Welcome to Gentoo/Linux!

[  OK  ] Reached target Remote File Systems.
[  OK  ] Listening on Delayed Shutdown Socket.
[  OK  ] Listening on /dev/initctl Compatibility Named Pipe.
[  OK  ] Reached target Paths.
[  OK  ] Listening on Journal Socket.
         Mounting Huge Pages File System...
         Mounting Debug File System...
         Mounting POSIX Message Queue File System...
         Mounting FUSE Control File System...
         Starting Journal Service...
[  OK  ] Started Journal Service.
[  OK  ] Reached target Swap.
         Starting Remount Root and Kernel File Systems...
         Mounting Temporary Directory...
[  OK  ] Started Remount Root and Kernel File Systems.
[  OK  ] Reached target Local File Systems (Pre).
         Starting Load Random Seed...
[  OK  ] Mounted Temporary Directory.
[  OK  ] Reached target Local File Systems.
         Starting Recreate Volatile Files and Directories...
         Starting Trigger Flushing of Journal to Persistent Storage...
[  OK  ] Mounted POSIX Message Queue File System.
[  OK  ] Mounted Debug File System.
[  OK  ] Mounted Huge Pages File System.
[  OK  ] Mounted FUSE Control File System.
[  OK  ] Started Load Random Seed.
[  OK  ] Started Trigger Flushing of Journal to Persistent Storage.
[  OK  ] Started Recreate Volatile Files and Directories.
[  OK  ] Reached target System Initialization.
[  OK  ] Reached target Timers.
[  OK  ] Listening on D-Bus System Message Bus Socket.
[  OK  ] Reached target Sockets.
[  OK  ] Reached target Basic System.
         Starting Login Service...
         Starting Permit User Sessions...
         Starting D-Bus System Message Bus...
[  OK  ] Started D-Bus System Message Bus.
[  OK  ] Started Permit User Sessions.
         Starting Console Getty...
[  OK  ] Started Console Getty.
[  OK  ] Reached target Login Prompts.
[  OK  ] Started Login Service.
[  OK  ] Reached target Multi-User System.
[  OK  ] Reached target Graphical Interface.


This is gentoo.

gentoo login: root
Password:
Last login: Mon Dec  9 13:58:42 CST 2013 on console
gentoo ~ # shutdown -h -P now

Broadcast message from root at gentoo (console) (Mon Dec  9 17:10:23 2013):

The system is going down for system halt NOW!
gentoo ~ # [  OK  ] Stopped target Graphical Interface.
[  OK  ] Stopped target Multi-User System.
[  OK  ] Stopped target Login Prompts.
         Stopping Console Getty...
         Stopping Login Service...
         Stopping D-Bus System Message Bus...
[  OK  ] Stopped Login Service.
[  OK  ] Stopped Console Getty.
[  OK  ] Stopped D-Bus System Message Bus.
         Stopping Permit User Sessions...
[  OK  ] Stopped Permit User Sessions.
[  OK  ] Stopped target Basic System.
[  OK  ] Stopped target Paths.
[  OK  ] Stopped target Timers.
[  OK  ] Stopped target Sockets.
[  OK  ] Closed /dev/initctl Compatibility Named Pipe.
[  OK  ] Closed Delayed Shutdown Socket.
[  OK  ] Closed D-Bus System Message Bus Socket.
[  OK  ] Stopped target System Initialization.
[  OK  ] Stopped target Swap.
[  OK  ] Stopped target Local File Systems.
         Unmounting /usr/portage...
         Unmounting /etc/resolv.conf...
         Unmounting /proc/sys/kernel/random/boot_id...
         Unmounting Temporary Directory...
         Stopping Journal Service...
[  OK  ] Stopped target Remote File Systems.
[  OK  ] Unmounted /usr/portage.
[  OK  ] Unmounted /etc/resolv.conf.
[  OK  ] Unmounted /proc/sys/kernel/random/boot_id.
[  OK  ] Unmounted Temporary Directory.
[  OK  ] Reached target Unmount All Filesystems.
[  OK  ] Stopped target Local File Systems (Pre).
         Stopping Remount Root and Kernel File Systems...
[  OK  ] Stopped Remount Root and Kernel File Systems.
[  OK  ] Reached target Shutdown.
Sending SIGTERM to remaining processes...
Sending SIGKILL to remaining processes...
Unmounting file systems.
Unmounting /sys/kernel/debug.
Unmounting /dev/mqueue.
Unmounting /sys/fs/fuse/connections.
Unmounting /dev/hugepages.
All filesystems unmounted.
Storage is finalized.
Container has been shut down.

>> Also, I usually need to machinectl terminate the machine, otherwise I
>> cannot start the same container with the same name (I think this is a
>> known bug).
>
> Yeah known bug, but when I played with kernel 3.12.3 today I didn't see
> it and started the same machine twice without any problem... So maybe
> it's fixed (or maybe I was just lucky!)

In my case still happens with 3.12.3.

>> Any idea why do this happens? I took a quick glance through nspawn.c
>> code, but the only read-only related code I've seen is when using bind
>> mounts, which I don't need to use to trigger this behavior.
>
> My guess is that somehow the events from inside the container are
> propigating back to the host and gentoo is doing some remount ro on
> shutdown.... just a guess tho'.

Thanks.
-- 
Canek Peláez Valdés
Posgrado en Ciencia e Ingeniería de la Computación
Universidad Nacional Autónoma de México