[systemd-devel] Journald stops logging when lots of output

[Lennart Poettering]

On Sat, 03.08.13 23:14, Pedro Francisco (pedrogfrancisco at gmail.com) wrote:

> (question migrated from test at lists.fedoraproject.org )
> 
> On Fedora 19, journald ( systemd-204-9.fc19.i686 ) stops logging. I
> had enabled lots of iwl3945 debugging ( various messages for each
> iwl3945 interaction -- modprobe iwl3945 debug=0x47ffffff ) throughout
> the night and the last thing `journalctl -b` has is:
> 
> Ago 03 02:25:45 s2 systemd-journal[29414]: Allowing system journal
> files to grow to 1.4G.
> Ago 03 02:25:45 s2 systemd-journal[29414]: Journal started
> Ago 03 02:25:45 s2 systemd[1]: systemd-journald.service: main process
> exited, code=exited, status=1/FAILURE
> Ago 03 02:25:45 s2 systemd[1]: Started Trigger Flushing of Journal to
> Persistent Storage.
> Ago 03 02:25:45 s2 systemd[1]: systemd-journald.service: main process
> exited, code=killed, status=10/USR1
> Ago 03 02:25:45 s2 systemd[1]: Started Trigger Flushing of Journal to
> Persistent Storage.
> Ago 03 02:25:45 s2 systemd[1]: systemd-journald.service: main process
> exited, code=killed, status=10/USR1
> Ago 03 02:25:45 s2 systemd[1]: Started Trigger Flushing of Journal to
> Persistent Storage.
> Ago 03 02:25:45 s2 systemd-journal[29419]: Allowing system journal
> files to grow to 1.4G.
> Ago 03 02:25:45 s2 systemd-journal[29419]: Journal started
> Ago 03 02:25:45 s2 systemd[1]: systemd-journald.service: main process
> exited, code=exited, status=1/FAILURE
> Ago 03 02:25:45 s2 systemd[1]: Started Trigger Flushing of Journal to
> Persistent Storage.
> Ago 03 02:25:45 s2 systemd-journal[29423]: Allowing system journal
> files to grow to 1.4G.
> Ago 03 02:25:45 s2 systemd-journal[29423]: Journal started
> Ago 03 02:25:45 s2 systemd[1]: systemd-journald.service: main process
> exited, code=exited, status=1/FAILURE
> Ago 03 02:25:45 s2 systemd[1]: Started Trigger Flushing of Journal to
> Persistent Storage.
> 
> It is now 10:35 and while dmesg has (lots) of new info, journalctl -f does not.
> 
> Any tips on what may be wrong?

Is this still a problem?

It seems journald dies for you for some reason. Is there a way how you
can reproduce the issue? If so, can you strace the journal daemon, then
reproduce it, and paste the last 100 strace lines somwhere before the
daemon exited? This should give us an indication what might have failed
that caused it to exit. 

(Of course, journald should not exit under any such circumstances, but
to find that we first need to track down why it does that currently).

Also, if you log using a tool like "logger", does that get lost too
after the issue happens?

Lennart

-- 
Lennart Poettering - Red Hat, Inc.