[systemd-devel] [RFC/PATCH 3/3] coredump: compress core files

[Zbigniew Jędrzejewski-Szmek]

On Wed, Jun 25, 2014 at 02:09:34PM +0200, Koen Kooi wrote:
> 
> Op 25 jun. 2014, om 10:59 heeft Lennart Poettering <lennart at poettering.net> het volgende geschreven:
> 
> > On Wed, 25.06.14 07:28, Zbigniew Jędrzejewski-Szmek (zbyszek at in.waw.pl) wrote:
> > 
> >> Hi,
> >> a POC patch to compress coredumps after they are written. In my simple
> >> tests, this brings huge savings, because the compressed coredump can
> >> compress to ~5%.
> >> 
> >> Unfortunately the core is first written uncompressed, then compressed
> >> by reading from disk and writing to the output file. This is ugly and
> >> slow, but I don't see a way around, if we want to get the backtrace
> >> without keeping everything in memory.
> > 
> > I have the suspicion that there indeed is no other nice way for
> > this... Hence looks good to me.
> > 
> >> 'coredumpctl gdb' is not implemented yet (the decompression part is
> >> missing).
> > 
> > Hmm, we probably should add a coredump.conf option for this to turn on.
> > 
> > Hmm, thinking about this, another idea might be to simply let the fs
> > solve this by adding a high-level coredump.conf option to turn on
> > FS_COMPR_FL for the coredump. This should do the right thing on btrfs
> > (afaics at least), but of course would leave other file systems out. But
> > then again, we develop the OS for tomorrow, not for yesterday here...
> > 
> > But then again, maybe this stuff shouldn't be exclusive. Maybe a new
> > coredump.conf option Compress=(filesystem|file|no) or so? (and maybe
> > even Compress=auto, which does the right thing automatically, depending
> > if the fs supports compression) After all, doing both file-level and
> > filesystem-level compression makes little sense...
> 
> The way btrfs currently does compression is to compress every (iirc 4k) block separately. Compressing the whole file at once will give a much bigger saving, especially when using xz. So btrfs/lzo will give you ~50% compression ratio vs the 5% mentioned above, a huge difference. I don't know how reiser4 or zfs tackle this.

In the light of this, I skipped the filesystem compression part.

I implemented the rest and pushed.

Zbyszek