[systemd-devel] Problems with systemd master

[Lennart Poettering]

On Fri, 14.02.14 15:23, Richard Purdie (richard.purdie at linuxfoundation.org) wrote:

> Hi,
> 
> I work on the Yocto Project and we've been having some stability issues
> with systemd based images on our automated testing infrastructure. We
> default to the last released version but it appears there have been a
> lot of changes in master so I tried the latest git master in the hope
> that this would at least show if the issue was know/fixed or not. The
> master version consistently fails with the same issue, I'm not sure if
> that is better or worse.
> 
> The logs show:
> 
> systemd[1]: Expecting device dev-ttyS0.device...
>          Expecting device dev-ttyS0.device...
> systemd[1]: job_get_timeout dev-ttyS0.device 1/96903843 0/18446744073709551615 -> 1/96903843
> [*     ] A start job is running for dev-ttyS0.device (15s / 1min 30s)
> systemd[1]: job_get_timeout dev-ttyS0.device 1/96903843 0/18446744073709551615 -> 1/96903843
> [ TIME ] Timed out waiting for device dev-ttyS0.device.
> [DEPEND] Dependency failed for Serial Getty on ttyS0.
> 
> Bisection shows this happens after this commit:
> 
> http://cgit.freedesktop.org/systemd/systemd/commit/src/libudev/libudev-monitor.c?id=df32a1caafe881fc7850a9b569179b877198685f
> (also modifed again in http://cgit.freedesktop.org/systemd/systemd/commit/src/libudev/libudev-monitor.c?id=e8a3b2dcfbd2caa047489b52f7c874833831e5ca)
> 
> If I comment out the if block:
> 
>  if (!udev_has_service(udev) && !udev_has_devtmpfs(udev)) {

Are you running this inside a container?

Note that the kernel does not virtualize devices currently, and thus one
cannot run udev in a container. Previously the containers would run in a
messy mix where they would get an incomplete /dev, that doesn't match
the /sys (which shows all real devices, but might exlcude network
devices handled by network namespaces) and which they should *never*
write to, and enumeration would not report any initialized devices, even
though udev notifications would send out inforation about how it just
initialized devices...

To make this more palatable we arranged things so that inside of the
container the /sys tree is available, though generally only mounted
read-only (that's at least what we suggest container managers to do, and
systemd-nspawn does that). Furthermore enumeration will work, and give
you a device list, but never report anything as initialized, and the
udev notifications are disabled, hence never pretending that
initialization completed either. Putting this together this means that
device handling is still available in the containers, but in a "degraded"
mode where nothing will ever appear as fully available. THis appeared
like the best we could do.

If people want proper devices in containers they should work with the
upstream kernel guys to virtualize them. However, we will not pretend
things could ever work without kernel support for this.

Normally, the check above should detect whether things are run in a
container: if /tmp is not a devtmps and if udevd is not available then
we assume this is a container. THis kind of check should be flexible
enough to automatically make things work should one day the kernel guys
make virtualization work and udev runs in the container.

> after bootup, the getty appears (daemon-reexec also does this).
> 
> I added some simple debugging and the thing that changes is that at
> bootup udev_has_service() looks for /run/udev/queue.bin and that doesn't
> exist at first. Later it does exist after rebuild_queue_file() runs and
> everything starts working.

Well, the queue.bin check may fail, as long as the devtmpfs check
doesn't fail. Are you not using devtmpfs?

Lennart

-- 
Lennart Poettering, Red Hat