[systemd-devel] heads-up: chasing journal(?) related regression in 219 causing boot hang/fail

[Didier Roche]

Le 20/02/2015 15:41, Michael Biebl a écrit :
> 2015-02-20 15:36 GMT+01:00 Martin Pitt <martin.pitt at ubuntu.com>:
>> Hello all,
>>
>> Since we updated to 219 in Ubuntu, several people reported boot
>> failures. Booting hangs a long time after starting D-Bus, in the
>> journal you get a lot of error messages like
>>
>>     systemd[1]: Failed to register match for Disconnected message: Connection timed out
>>     systemd-logind[749]: Failed to fully start up daemon: Connection timed out
>>     dbus[800]: [system] Failed to activate service 'org.freedesktop.PolicyKit1': timed out
>>
>> polkitd isn't running. This causes lots of jobs (logind, NetworkManager, avahi,
>> etc.) to get stuck in an eternal retry loop.
>>
>> Unfortunately reproducing this is a real nuisance, classic heisenbug.
>> I'm now able to trigger it (sometimes) in a VM, but I still haven't
>> found a reliable recipe for reproducing it, so that bisecting just
>> takes ages.
>>
>> I'm keeping debug log, notes, and progress in
>> https://launchpad.net/bugs/1423811 FTR. This is mostly a heads-up for
>> other distros in case they also get reports like this, to shortcut the
>> debugging exercise (I already wasted 7 hours on this, and I'm not even
>> close to the solution). Quite surprisingly it's somewhere in journald.
>> Running 218 with journald from 219 causes the hang, 219 with journald
>> from 218 is fine.
>>
> I noticed this as well. Interestingly, it only ever happened after
> applying the fsckd patches  and running with plymouth enabled.
>

We get it with "quiet splash" removed as well. It just that it's random 
on the machine load. We already ruled out the fsckd patch yesterday, but 
I did retry today again after this comment on my own vms and with 
systemd/udev/… ubuntu package "218-8ubuntu2" which doesn't contain the 
fsckd patch (and have the 218 and 219 systemd-fsck writing to 
/dev/console) + systemd-journal binary copied from 219-1ubuntu1, I was 
able to reproduce the hang after 15 boots.

Cheers,
Didier