[systemd-devel] xz troubles

[Zbigniew Jędrzejewski-Szmek]

Hi,

this is in some sense a continuation of the subject of xz compression
for coredumps.

I've been benchmarking systemd-journal-remote + nc upload speed on
some fake data in which the MESSAGE fields are a few kb, big enough to
trigger xz compression in the journal. This is probably not a normal
stream, but not a completely impossible one. The result is of sending
708 MB of logs is:
- without compression, 713 MB of journal files, 23 s,
- with compression, 361 MB of journal, 19 min 29 s!

So the 23 seconds are not particularly impressive, I think there are
some unnecessary reallocs and copies there which can be eliminated
improved by implementing a sliding window. I tried to minimize the
memory usage, but it seems to be hurting performance. But the 19.5
minutes is rather bad, and I think it is attributable to compression
of a gazillion relatively small blobs.

This could also explain why journald sometimes performes so badly.

A different compressor?

Zbyszek