[systemd-devel] xz troubles

[Tom Gundersen]

On Wed, Jul 2, 2014 at 2:20 AM, Zbigniew Jędrzejewski-Szmek
<zbyszek at in.waw.pl> wrote:
> Hi,
>
> this is in some sense a continuation of the subject of xz compression
> for coredumps.
>
> I've been benchmarking systemd-journal-remote + nc upload speed on
> some fake data in which the MESSAGE fields are a few kb, big enough to
> trigger xz compression in the journal. This is probably not a normal
> stream, but not a completely impossible one. The result is of sending
> 708 MB of logs is:
> - without compression, 713 MB of journal files, 23 s,
> - with compression, 361 MB of journal, 19 min 29 s!
>
> So the 23 seconds are not particularly impressive, I think there are
> some unnecessary reallocs and copies there which can be eliminated
> improved by implementing a sliding window. I tried to minimize the
> memory usage, but it seems to be hurting performance. But the 19.5
> minutes is rather bad, and I think it is attributable to compression
> of a gazillion relatively small blobs.
>
> This could also explain why journald sometimes performes so badly.
>
> A different compressor?

Seems reasonable to try to use one that is built for fast
compression/decompression speeds. I guess xz is only really good for
"offline" compression, where compression rate and decompression speed
is all that matters.

-t