[systemd-devel] Can coredumpctl work independent of journald?

[P.R.Dinesh]

Thank you Lennart,
I would like to explain my system scenario.

We are using systemd version 219. (Updating to 229 is in progress).

Configured for persistent storage for both Journal and Coredump (Coredump
is stored externallly)

The logs and coredump are stored in another partition
"/var/diagnostics/logs" and "/var/diagnostics/coredump".  We do symbolic
link between the /var/lib/systemd/coredump and logs to those folders.

Coredump and journald is configured to utilized upto 10% of the disk
space(total disk space is ~400MB) which would allocate 40MB to journal logs
and 40MB to coredump.  For some reason (under investigation) some of our
daemons are generating too much logs which makes the journald to reach the
40MB limit within 6 hours.  Hence journald starts wrapping around.
Meanwhile some daemons have also crashed and core dumped.

Now when I do coredumplist, none of those coredumps are shown.

Also I tried launching the coredumpctl with the coredump file both using
 the pid name as well as using the coredump file name.  Since we dont have
the journal entry coredumpctl is not launching them,  can we atleast have
the coredumpctl launch the gdb using the core dump file name?

[prd at localhost ~]$ coredumpctl gdb
core.lshw.1000.9bb41758bba94306b39e751048e0cee9.23993.1462871523000000.xz
No match found.
[prd at localhost ~]$ coredumpctl gdb 23993
No match found.


In summary, the frequency of logs are higher and the frequency of core
dumps are very less in our system which leads to the loss of coredump
information.

I am thinking of two solutions here
1) Enhance coredumpctl to launch the gdb using the coredump file name
2) Store the Journal logs for coredump seperately from other journal logs
so that they could be maintained for long duration (is this feasible?)

Thank you
Regards,
Dinesh

On Wed, May 11, 2016 at 10:25 PM, Lennart Poettering <lennart at poettering.net
> wrote:

> On Wed, 11.05.16 20:31, P.R.Dinesh (pr.dinesh at gmail.com) wrote:
>
> > I have set the journald to be persistant and limit its size to 40MB.
> > I had a process coredumped and the coredump file is found in
> > /var/log/systemd/coredump
> >
> > When I run coredumpctl this coredump is not shown.
> >
> > Later I found that the core dump log is missing from the Journal ( the
> > journal got wrapped since it reached the size limitation).
> >
> > I think coredumpctl depends on journal to display the coredump.  Can't it
> > search for the coredump files present in the coredump folder and list
> those
> > files?
>
> We use the metadata and the filtering the journal provides us with,
> and the coredump on disk is really just secondary, external data to
> that, that can be lifecycled quicker than the logging data. We extract
> the backtrace from the coredump at the momemt the coredump happens,
> and all that along with numerous metadata fields is stored in the
> journal. In fact storing the coredump is optional, because in many
> setups the short backtrace in the logs is good enough, and the
> coredump is less important.
>
> So, generally the concept here really is that logs are cheap, and thus
> you keep around more of them; and coredumps are large and thus you
> lifecycle them quicker. If I understand correctly what you want is the
> opposite: you want a quicker lifecycle for the logs but keep the
> coredumps around for longer. I must say, I am not entirely sure where
> such a setup would be a good idea though... i.e. wanting persistent
> coredumps but volatile logging sounds a strange combination to
> me... Can you make a good case for this?
>
> But yeah, we really don't cover what you are asking for right now, and
> I am not sure we should...
>
> > Also can I launch the coredumpctl gdb by providing a compressed core
> > file.
>
> If you configure systemd-coredump to store the coredumps compressed
> (which is in fact the default), then "coredumpctl gdb" will implicitly
> decompress them so that gdb can do its work.
>
> Lennart
>
> --
> Lennart Poettering, Red Hat
>



-- 
With Kind Regards,
Dinesh P Ramakrishnan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/systemd-devel/attachments/20160512/77ad8a97/attachment.html>