[systemd-devel] Confusing journal information - journal size

Mon Jul 20 08:49:11 PDT 2015

David Sommerseth wrote on 20/07/15 16:29:
> On 20/07/15 15:31, Anne Mulhern wrote:
>> ----- Original Message -----
>>> From: "Colin Guthrie" <gmane at colin.guthr.ie>
>>> To: systemd-devel at lists.freedesktop.org
>>> Sent: Monday, July 20, 2015 5:38:38 AM
>>> Subject: Re: [systemd-devel] Confusing journal information - journal size
>>>
>>> David Sommerseth wrote on 17/07/15 14:28:
>>>> On 17/07/15 13:31, Mantas Mikulėnas wrote:
>>>>> On Fri, Jul 17, 2015 at 2:13 PM, David Sommerseth <davids at redhat.com
>>>>> <mailto:davids at redhat.com>> wrote:
>>>>>
>>>>>
>>>>>     Hi,
>>>>>
>>>>>     I'm looking through some journals now, and even though I've seen it a
>>>>>     few times I haven't thought about it until now.
>>>>>
>>>>>        systemd-journal[1151]: Runtime journal is using 8.0M (max allowed
>>>>>                  4.0G, trying to leave 4.0G free of 63.7G available →
>>>>>                  current limit 4.0G).
>>>>>
>>>>>     Could this line be cleaned up so you don't have to look up a man page
>>>>>     to
>>>>>     try to figure out what this really means?  Here's my uneducated guess
>>>>>     and confusion of this line:
>>>>>
>>>>>     * Runtime journal is using 8.0M
>>>>>       - Okay, so currently the journal uses 8MB of disk-space.  No
>>>>>       problem.
>>>>>
>>>>>     * max allowed 4.0G
>>>>>       - Okay, so the journal should not grow beyond 4GB, makes sense.  No
>>>>>         problem.
>>>>>
>>>>>     * trying to leave 4.0G free of 63.7G available
>>>>>       - Uhm, what!? So it will grow until there is 4GB left on the
>>>>>         filesystem?  Not so okay.
>>>>>
>>>>>
>>>>> It chooses the /smallest/ limit, not largest. (Common sense...) For
>>>>> example, if you had only 5 GB space available, the journal would not
>>>>> grow beyond 1 GB.
>>>>>  
>>>>>
>>>>>     * current limit 4.0G
>>>>>       - Ehh ... okay ... so make up your mind, please!  So will the
>>>>>         journal grow until 4GB or 59.7GB.
>>>>>
>>>>>
>>>>> This *is* it making up its mind: "min(limit 1, limit 2) → resulting limit"
>>>>>
>>>>>     But then I looked into /var/log/journal ...
>>>>>
>>>>>       # du --si -s /var/log/journal/
>>>>>       4.3G  /var/log/journal/
>>>>>
>>>>>     I do see that both system,journal and user-UID.journal are both 8.4MB,
>>>>>     and from that I can guess what the log entry tried to tell me with
>>>>>     "Runtime journal" ... but how is /that/ information useful for me,
>>>>>     from
>>>>>     a sys-admin point of view?
>>>>>
>>>>>
>>>>> "Runtime" here means /run, as opposed to persistent in /var. They have
>>>>> separately configurable limits, since /run is in RAM and /var is usually
>>>>> on disk. (Though, I'm not entirely sure what purpose the runtime journal
>>>>> even serves, when /var is available.)
>>>>
>>>> Fair enough.  But you are missing my point.
>>>>
>>>> How this information is presented do require some detail knowledge of
>>>> the journal.  Don't think like a developer who have poked at the journal
>>>> code.  Think like a sys-admin who looks through the logs looking for
>>>> issues.  Then you want to have the answer straight in your face, not
>>>> needing to go elsewhere to read about these things.  In fact most admins
>>>> will probably have forgotten what they were going to look for when they
>>>> move their eyes of the log data.
>>>>
>>>> If it is considered important information, fine.  But present it in a
>>>> far more understandable way for those who just uses the journal.  Right
>>>> now, I'm not surprised if most sys-admins read that line as useless
>>>> gibberish - "Yeah, yeah, journal will waste some space on my drive".
>>>
>>> Yeah, I can't disagree with David. Not sure how best to tidy it up, but
>>> some rework would definitely be nice.
>>>
>>> Col
>>>
>>
>> After seeing the explanation, the best complete and correct (AFAICT) formulation I could come up with was,
>>
>> "Runtime journal is using 8.0M (max allowed = min(4.0G, S s.t. total memory(63.7 G) - S = 4.0 G (59.7 G), available memory (16.2 G)) = 4.0G)"
>>
>> which is compelled to use math speak for clarity and succinctness.
>>
>> Dunno how happy most sys-admins would be with that.
>>
>> - mulhern
> 
> But is all that information really needed?
> 
> If I try to see this from a sys-admin point of view there are two
> numbers which are important to me: 1) Current state 2) Final journal
> limit size.  From how I see it, how the journal code ends up with a
> certain number is only useful when you're developing/debugging the
> journal.  Remember: Less is more.

Well I guess you could just log something like:

"Runtime journal is using 8.0M (see 'journalctl status' for more info)"

Then you add a "journalctl status" verb that explained the current
status of journal (e.g. number of files on disk and in memory, how the
file size and rotation will work etc)

That might be more practically useful, but it won't explain things as
calculated at the time that log entry was created, so can I suggest that
an additional "_CALCULATION" field (or soemthing similarly named) is
added into that log message that is not shown by default but is stored)
so that the typical administrator looking at the log out put will not
see the detail, but it is logged.

The journalctl status command could even pull out the last messages in
the journal (via it's message id) and then get the _CALCULATION field
and show it in a nice, verbose way to the user.

That keeps it simple by default but has all the juicy details there
should they be needed.

Just a thought.

Col

-- 

Colin Guthrie
colin(at)mageia.org
http://colin.guthr.ie/

Day Job:
  Tribalogic Limited http://www.tribalogic.net/
Open Source:
  Mageia Contributor http://www.mageia.org/
  PulseAudio Hacker http://www.pulseaudio.org/
  Trac Hacker http://trac.edgewall.org/