[systemd-devel] journal space requirements

[Lennart Poettering]

On Thu, 29.11.12 10:37, Colin Guthrie (gmane at colin.guthr.ie) wrote:

> Hi,
> 
> So a couple complaints/queries are beginning trickling in regarding
> journal space requirements.
> 
> A user was complaining that rotated journals were taking up too much
> room and they could be compressed down etc. I did explain that a rotated
> journal is really any different to the current journal other than it can
> be sealed, but I do fear he has some kind of point regarding long term
> storage.
> 
> Should there be some kind of journal archiver system that will run xz -9
> on older journals? Is this something that's being planned or is it left
> as an exercise for the reader to implement such long term
> storage/archiving systems?
> 
> IMO it would be nice to be able to trigger such service when the journal
> is rotated (i.e. a templated unit) that can either just compress the
> journal in place (assuming the user does not want to be able to read
> from it actively any more) or copy a compressed version to some archive
> dir to do with as the user pleases.

As Kay already pointed out it's a bit hard to fully compress the files
including the metadata if we still want to allow implicit efficient
merging of all accessible journal files rotated or not. It's the
tradeoff: either you index everything and can access it very
efficiently, so that it becomes so cheap you can just do it implicitly
and always, or you reach the best possible compression...

I think we have a couple of other options though. For example, it has
been suggested we should drop lower-prio msgs when rotating. I think it
would make a lot of sense to implement something like this on rotation,
i.e. where we just rewrite the entire file dropping all debug messages,
so that as older things get the less detail we maintain...

Lennart

-- 
Lennart Poettering - Red Hat, Inc.