[systemd-devel] Systemd with an initramfs

[Ian Geiser]

Greetings, I am working on an embedded system where I load the users pace rootfs off of a loop device.  I then use AUFS to put tmpfs overlay on it and then switch root into that.  Currently systemd works like a champ getting through the initramfs.  It correctly mounts the loop device, tmpfs and the aufs.  I then also move the mounts for the loop filesystem and tmpfs into the /sysroot.  This is where things start to have problems.  While I can boot successfully into multi-user there is an error on boot about not being able to unmount the loop back mount.  

 Jan 01 00:00:43 raspberrypi0 umount[215]: umount: can't umount /lib/live/rofs: Device or resource busy

Of course it is busy because its hosting the userspace rootfs.  Then further down I see these errors:

Jan 01 00:00:43 raspberrypi0 systemd[1]: lib-live-rofs.mount: Unit is bound to inactive unit dev-loop0.device. Stopping, too.
Jan 01 00:00:43 raspberrypi0 systemd[1]: lib-live-rofs.mount: Installed new job lib-live-rofs.mount/stop as 302
Jan 01 00:00:43 raspberrypi0 systemd[237]: lib-live-rofs.mount: Executing: /bin/umount /lib/live/rofs
Jan 01 00:00:43 raspberrypi0 systemd[1]: Received SIGCHLD from PID 237 (umount).
Jan 01 00:00:43 raspberrypi0 systemd[1]: lib-live-rofs.mount: Changed unmounting -> mounted
Jan 01 00:00:43 raspberrypi0 systemd[1]: lib-live-rofs.mount: Installed new job lib-live-rofs.mount/stop as 303
Jan 01 00:00:44 raspberrypi0 systemd[1]: lib-live-rofs.mount: About to execute: /bin/umount /lib/live/rofs
Jan 01 00:00:44 raspberrypi0 systemd[239]: lib-live-rofs.mount: Executing: /bin/umount /lib/live/rofs
Jan 01 00:00:44 raspberrypi0 systemd[1]: lib-live-rofs.mount: Child 239 belongs to lib-live-rofs.mount
Jan 01 00:00:44 raspberrypi0 systemd[1]: lib-live-rofs.mount: Trying to enqueue job lib-live-rofs.mount/stop/fail
Jan 01 00:00:44 raspberrypi0 systemd[1]: lib-live-rofs.mount: Installed new job lib-live-rofs.mount/stop as 304
Jan 01 00:00:44 raspberrypi0 systemd[241]: lib-live-rofs.mount: Executing: /bin/umount /lib/live/rofs

.... about 3x of this

Jan 01 00:00:46 raspberrypi0 systemd[1]: lib-live-rofs.mount: About to execute: /bin/umount /lib/live/rofs
Jan 01 00:00:46 raspberrypi0 systemd[1]: lib-live-rofs.mount: Changed mounted -> unmounting
Jan 01 00:00:46 raspberrypi0 systemd[1]: Unmounting /lib/live/rofs...
Jan 01 00:00:46 raspberrypi0 systemd[258]: lib-live-rofs.mount: Executing: /bin/umount /lib/live/rofs
Jan 01 00:00:46 raspberrypi0 systemd[1]: Received SIGCHLD from PID 258 (umount).
Jan 01 00:00:46 raspberrypi0 systemd[1]: Child 258 (umount) died (code=exited, status=1/FAILURE)
Jan 01 00:00:46 raspberrypi0 systemd[1]: lib-live-rofs.mount: Child 258 belongs to lib-live-rofs.mount
Jan 01 00:00:46 raspberrypi0 systemd[1]: lib-live-rofs.mount: Mount process exited, code=exited status=1
Jan 01 00:00:46 raspberrypi0 systemd[1]: lib-live-rofs.mount: Changed unmounting -> mounted
Jan 01 00:00:46 raspberrypi0 systemd[1]: lib-live-rofs.mount: Job lib-live-rofs.mount/stop finished, result=failed
Jan 01 00:00:46 raspberrypi0 systemd[1]: Failed unmounting /lib/live/rofs.
Jan 01 00:00:46 raspberrypi0 systemd[1]: lib-live-rofs.mount: Unit is bound to inactive unit dev-loop0.device. Stopping, too.
Jan 01 00:00:46 raspberrypi0 systemd[1]: lib-live-rofs.mount: Trying to enqueue job lib-live-rofs.mount/stop/fail
Jan 01 00:00:46 raspberrypi0 systemd[1]: lib-live-rofs.mount: Installed new job lib-live-rofs.mount/stop as 312
Jan 01 00:00:46 raspberrypi0 systemd[1]: lib-live-rofs.mount: Enqueued job lib-live-rofs.mount/stop as 312
Jan 01 00:00:46 raspberrypi0 systemd[1]: lib-live-rofs.mount: About to execute: /bin/umount /lib/live/rofs
Jan 01 00:00:46 raspberrypi0 systemd[1]: lib-live-rofs.mount: Forked /bin/umount as 260
Jan 01 00:00:46 raspberrypi0 systemd[1]: lib-live-rofs.mount: Changed mounted -> unmounting
Jan 01 00:00:46 raspberrypi0 systemd[1]: Unmounting /lib/live/rofs...

....  about 5x of this

Jan 01 00:00:47 raspberrypi0 systemd[1]: lib-live-rofs.mount: About to execute: /bin/umount /lib/live/rofs
Jan 01 00:00:47 raspberrypi0 systemd[1]: lib-live-rofs.mount: Forked /bin/umount as 270
Jan 01 00:00:47 raspberrypi0 systemd[1]: lib-live-rofs.mount: Changed mounted -> unmounting
Jan 01 00:00:47 raspberrypi0 systemd[1]: Unmounting /lib/live/rofs...
Jan 01 00:00:47 raspberrypi0 systemd[270]: lib-live-rofs.mount: Executing: /bin/umount /lib/live/rofs
Jan 01 00:00:47 raspberrypi0 umount[253]: umount: can't umount /lib/live/rofs: Device or resource busy
Jan 01 00:00:47 raspberrypi0 umount[255]: umount: can't umount /lib/live/rofs: Device or resource busy
Jan 01 00:00:47 raspberrypi0 umount[258]: umount: can't umount /lib/live/rofs: Device or resource busy
Jan 01 00:00:47 raspberrypi0 umount[260]: umount: can't umount /lib/live/rofs: Device or resource busy
Jan 01 00:00:47 raspberrypi0 umount[262]: umount: can't umount /lib/live/rofs: Device or resource busy
Jan 01 00:00:47 raspberrypi0 systemd-cgroups-agent[271]: Failed to determine peer security context: Protocol not available
Jan 01 00:00:47 raspberrypi0 systemd[1]: Received SIGCHLD from PID 270 (umount).
Jan 01 00:00:47 raspberrypi0 systemd[1]: Child 270 (umount) died (code=exited, status=1/FAILURE)
Jan 01 00:00:47 raspberrypi0 systemd[1]: lib-live-rofs.mount: Child 270 belongs to lib-live-rofs.mount
Jan 01 00:00:47 raspberrypi0 systemd[1]: lib-live-rofs.mount: Mount process exited, code=exited status=1
Jan 01 00:00:47 raspberrypi0 systemd[1]: lib-live-rofs.mount: Changed unmounting -> mounted
Jan 01 00:00:47 raspberrypi0 systemd[1]: lib-live-rofs.mount: Job lib-live-rofs.mount/stop finished, result=failed
Jan 01 00:00:47 raspberrypi0 systemd[1]: Failed unmounting /lib/live/rofs.
Jan 01 00:00:47 raspberrypi0 systemd[1]: lib-live-rofs.mount: Unit is bound to inactive unit dev-loop0.device, but not stopping since we tried this too often recently.

If I look in "/dev" I do see that the loop devices are still there.  Is there something that I did wrong before I started the cleanup process?

I have linked the full log at: https://oc.geekcentral.pub/index.php/s/2dQtWOdojb6srlE but my spidey senses tell me I forgot a step in the initramfs process that is causing the /dev to get cleared out and then rebuilt in userspace.  Does this sound like a correct path of reasoning?

Thanks!