[systemd-devel] systemd hang on reboot

[Lennart Poettering]

On Thu, 29.03.12 18:58, Albert Strasheim (fullung at gmail.com) wrote:

> Hello all
> 
> We have a problem with machines running systemd-37-15.fc16.x86_64
> hanging on reboot.
> 
> We have a system that boots from an initramfs based on F16. Nothing
> much more special than yum --installroot and making a cpio archive out
> of it.
> 
> We use the following kernel options that might be pertinent:
> 
> plymouth.enable=0
> readonlyroot
> systemd.log_level=debug
> systemd.log_target=kmsg
> systemd[1]: Child 8066 belongs to console-kit-log-system-stop.service
> systemd[1]: console-kit-log-system-stop.service: main process exited,
> code=exited, status=0
> systemd[1]: console-kit-log-system-stop.service changed start -> dead
> systemd[1]: Job console-kit-log-system-stop.service/start finished, result=done
> systemd[1]: Accepted connection on private bus.
> systemd[1]: Accepted connection on private bus.
> systemd[1]: Got D-Bus request:
> org.freedesktop.systemd1.Agent.Released() on
> /org/freedesktop/systemd1/agent
> systemd[1]: console-kit-log-system-stop.service: cgroup is empty
> systemd[1]: Got D-Bus request:
> org.freedesktop.systemd1.Agent.Released() on
> /org/freedesktop/systemd1/agent
> systemd[1]: network.service: cgroup is empty
> systemd[1]: Got D-Bus request:
> org.freedesktop.DBus.Local.Disconnected() on
> /org/freedesktop/DBus/Local
> systemd[1]: Got D-Bus request:
> org.freedesktop.DBus.Local.Disconnected() on
> /org/freedesktop/DBus/Local
> 
> What is supposed to happen next in the shutdown process? Is there any
> way we can get more debugging info to find out why it is hanging?

If debug mode is enabled systemd will dump the transaction it is about
to execute before it executes it. You should look for that output
between "Trying to enqueue...." till "Enqueued..." for the specific
shutdown target you are using (poweroff.target most likely).

Then, strike of that list everything that is successfully shutdown and
you should have an idea what is remaining.

Other than that you can simply run openvt during normal operation, to
get a shell on a new VT, and then use "echo $$ > /sys/fs/cgroup/systemd/tasks"
to escape systemd's supervision. Then use this shell to explore what is
going on, what is still running, and what is still queued (systemctl
list-jobs for example).

Lennart

-- 
Lennart Poettering - Red Hat, Inc.