[systemd-devel] 220 udev boot regression: timeout, giving up waiting for workers to finish

[Tom Gundersen]

On Tue, May 26, 2015 at 9:04 PM, Christian Hesse <list at eworm.de> wrote:
> Martin Pitt <martin.pitt at ubuntu.com> on Tue, 2015/05/26 17:11:
>> Hello Tom, all,
>>
>> with 220 I get a severe boot time regression:
>>
>>   $ systemd-analyze
>>   Startup finished in 30.751s (kernel) + 11.706s (userspace) = 42.458s
>>
>> which used to be
>>
>>   $ systemd-analyze
>>   Startup finished in 703ms (kernel) + 890ms (userspace) = 1.593s
>>
>> (this is a VM)
>>
>> It seems udevd --daemon spends 30 seconds timing out in the initramfs:
>>
>>   [    0.384519] systemd-udevd[55]: starting version 220
>>   [   30.736381] systemd-udevd[56]: timeout, giving up waiting for workers
>> to finish
>>
>> and then some more in the real root:
>>
>>    $ systemd-analyze blame
>>      10.826s dev-vda1.device
>>      10.067s systemd-tmpfiles-setup-dev.service
>>      10.031s systemd-sysctl.service
>>      10.019s systemd-journald.service
>>      10.005s sys-fs-fuse-connections.mount
>>      10.001s tmp.mount
>>
>> (full journal at http://paste.ubuntu.com/11372265/, but it's not very
>> useful)
>>
>> I bisected this to
>>
>>   http://cgit.freedesktop.org/systemd/systemd/commit/?id=e237d8c
>>   udevd: move file descriptors to Manager
>>
>> this is hard to revert individually as there are lots of other recent
>> changes in udev around this commit, but any version before that commit is
>> fast and doesn't give that timeout error.
>>
>> Current trunk as of commit 185abfc3 still has that problem, so it
>> wasn't fixed by one of the recent udev commits.
>>
>> Does anyone else see this too? Any idea what causes this?
>
> I do see this as well. And probably we have an upstream bug [0] already.

Oh, and this bug report is probably unrelated. It is an old one (that
I could never reproduce), and the relevant parts were reworked
(probably causing the current issues).

-t

> Wondering whether or not my report about "inotify_add_watch() failed: Bad
> file descriptor" [1] is related. Do you see that as well?
> BTW, is it expected to have fd_inotify in udevd.c and inotify_fd in
> udev_watch.c?
>
> [0] https://bugs.freedesktop.org/show_bug.cgi?id=90051
> [1] http://lists.freedesktop.org/archives/systemd-devel/2015-May/032213.html
> --
> main(a){char*c=/*    Schoene Gruesse                         */"B?IJj;MEH"
> "CX:;",b;for(a/*    Chris           get my mail address:    */=0;b=c[a++];)
> putchar(b-1/(/*               gcc -o sig sig.c && ./sig    */b/42*2-3)*42);}