[systemd-devel] journal space requirements

[Colin Guthrie]

'Twas brillig, and Colin Guthrie at 29/11/12 12:22 did gyre and gimble:
> 'Twas brillig, and Zbigniew Jędrzejewski-Szmek at 29/11/12 11:48 did
> gyre and gimble:
>> On Thu, Nov 29, 2012 at 10:37:26AM +0000, Colin Guthrie wrote:
>>> Hi,
>>>
>>> So a couple complaints/queries are beginning trickling in regarding
>>> journal space requirements.
>>>
>>> A user was complaining that rotated journals were taking up too much
>>> room and they could be compressed down etc. I did explain that a rotated
>>> journal is really any different to the current journal other than it can
>>> be sealed, but I do fear he has some kind of point regarding long term
>>> storage.
>>>
>>> Should there be some kind of journal archiver system that will run xz -9
>>> on older journals? Is this something that's being planned or is it left
>>> as an exercise for the reader to implement such long term
>>> storage/archiving systems?
>>
>> systemd-journal-remote can be used (when merged) to rewrite journal
>> files, so the tools are mostly there.  Nevertheless, the data in
>> .journal files should already be compressed with XZ (if enabled), so I
>> doubt that that would gain much. Do you have some statistics about
>> the gains from the additional external XZ compression?
> 
> Hmmm, I wonder if I've missed something obvious in my build then.
> 
> I just took a cleanly rotated journal of 18M in size and it compressed
> down to 4M... might be time to look at the build requires :D

OK, so I think I probably was missing an XZ build dep, but even on a
local build (where I have the devel libs for it already installed), it
still seems to make a big difference...

-rw-r----- 1 root adm  4.7M Nov 29 12:36 system.journal
-rw-r--r-- 1 root root 132K Nov 29 12:37 system.journal.xz


Am I missing something obvious here. Is "XZ...yes" in the configure
output enough or do I have to do something special to enable it in the
journal too?

Or is it just that the uncompressed indexes take up that much space
generally or that on-the-fly compression just isn't as efficient as post
rotation compression?

Col

-- 

Colin Guthrie
gmane(at)colin.guthr.ie
http://colin.guthr.ie/

Day Job:
  Tribalogic Limited http://www.tribalogic.net/
Open Source:
  Mageia Contributor http://www.mageia.org/
  PulseAudio Hacker http://www.pulseaudio.org/
  Trac Hacker http://trac.edgewall.org/