[systemd-devel] systemd-journald missing crash logs

[guenther kuenzel]

the kernel itself has a remote dump functionality. this might be your only option when your system is not even able to write logs anymore.

https://help.ubuntu.com/lts/serverguide/kernel-crash-dump.html

greetings


Am 22.01.18 um 04:53 schrieb Farzad Panahi:
> Hi Lennart - Thanks for your comments.
>  
> On Mon, Jan 15, 2018 at 4:11 AM, Lennart Poettering <lennart at poettering.net <mailto:lennart at poettering.net>> wrote:
>
>     On Fr, 12.01.18 16:13, Farzad Panahi (farzad.panahi at gmail.com <mailto:farzad.panahi at gmail.com>) wrote:
>
>     > I am running Arch-ARM on RPi3. I have noticed when system crashes I cannot
>     > find any related crash log in journal logs.
>
>     What specifically is a "crash" supposed to mean?
>
>
> Crash in my case means that the box becomes unresponsive. Meaning that I cannot ssh to it anymore until it is power cycled. I do not know what is happening to the box because there are no logs at the time of crash. Logs start rolling after the reboot. 
>
>     journald syncs to disk whenever a log message above LOG_ERR is
>     delivered. I am not sure what "crash" is supposed to mean, but are you
>     sure that at least one LOG_CRIT/LOG_ALERT/LOG_EMERG message is
>     delivered to userspace about that?
>
> I am not sure about that. I just assume if some critical issue is happening such that it makes the system unresponsive, then there should be high priority logs associated with it. 
>
>     > > Since the syslog component of systemd, journald, does not flush its
>     > > logs to disk during normal operation, these logs will be gone when the
>     > > machine is shut down abnormally (power loss, kernel lock-ups, ...). In
>     > > the case of kernel lock-ups, it is pretty important to have some
>     > > kernel logs for debugging. Until journald gains a configuration option
>     > > for flushing kernel logs, rsyslog can be used in conjunction with
>     > > journald.
>
>     As mentioned above, we wil sync immediately when a
>     LOG_CRIT/LOG_ALERT/LOG_EMERG log message is seen. We'll also sync on
>     normal log messages with a delay of 5min at max:
>
>     https://github.com/systemd/systemd/blob/master/src/journal/journald-server.c#L1440 <https://github.com/systemd/systemd/blob/master/src/journal/journald-server.c#L1440>
>
>     if you get get a hard kernel lockup for some reason then this all is
>     useless however, as userspace won't even get the opportunity to write
>     anything to disk then... And it doesn't matter if userspace runs
>     journald or rsyslog.
>
>
> So I think one of the following is happening: 
> a) no log is generated at the time of crash (I think this is unlikely) 
> b) log is generated but does not reach journald 
> c) log reaches journald but journald does not get a chance to persist it
> d) journald persists the log but somehow the log is corrupted and ignored
>
> I think scenario "c" is the most probable one in my case. So I just want to confirm if kernel panics, then most probably I will not see any logs in my log files? Is there a recommended workaround to debug such cases? 
>
>
>
> _______________________________________________
> systemd-devel mailing list
> systemd-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/systemd-devel