[systemd-devel] Trying to decipher a patch made in Centos 7 RPMs

[Mantas Mikulėnas]

On Wed, Sep 25, 2019 at 6:32 AM Robert Winslow Dalpe <rwdalpe at gmail.com>
wrote:

> Hello,
>
> I'm trying to track down the reason for an issue I'm seeing between
> systemd-journald and how rsyslog where data is being truncated when read
> from the journal file ( https://github.com/rsyslog/rsyslog/issues/3784 ).
>
> The behavior seems present in systemd 219 (on amazon linux 2 and centos 7)
> and not in later versions (on arch linux). I figured the truncation may be
> due to the default threshold for sd_journal_get_data (
> https://www.freedesktop.org/software/systemd/man/sd_journal_get_data.html
> ). From what I can tell, it looks like the only time sd_journal_get_data
> uses the threshold is when calling `decompress_blob` (
> https://github.com/systemd/systemd/blob/a88abde72169ddc2df77df3fa5bed30725022253/src/journal/sd-journal.c#L1916
> ). `decompress_blob` looks to see if the data is compressed using XZ or LZ4
> and then decompresses appropriately (
> https://github.com/systemd/systemd/blob/a88abde72169ddc2df77df3fa5bed30725022253/src/journal/compress.c#L214
> ). It looks like only the XZ decompression code utilizes the threshold
> (`dst_max`--which contains the threshold value--is unused in
> `decompress_blob_lz4`).
>
> From what I can tell LZ4 compression is the default. Assuming I'm on the
> right track, I'm trying to determine how XZ decompression would be invoked.
> That led me to the RPM that centos7 (and thus presumably amazon linux 2)
> builds for systemd. That RPM applies the following patch:
> https://git.powerel.org/powerel7/base/src/commit/e9d94968812dc1b501e6c691ee8c01595d77f76d/SOURCES/0575-Add-support-to-read-lz4-compressed-journals.patch
>  Although the patch says it "adds support to read lz4 compressed journals,"
> it looks to me like it is actually making XZ compression the default by
> removing several preprocessor directives that would allow for LZ4
> compression/decompression. Am I interpreting that patch correctly? Does any
> of what I've said seem even close to sane? I realize I can remove that
> patch and try building an RPM myself, but I'm hoping someone might know
> enough to provide any quick feedback before I go down that road.
>

Without the patch, XZ is already the default. The patch adds LZ4
decompression support, but tries to avoid making LZ4 the *default* format
for compression, making sure that e.g. compress_lz4 remains NULL despite
the LZ4 library being present. (I'm guessing one reason is that the LZ4
library already changed its framing format once, and CentOS wants to stick
with the stable xz format?)

-- 
Mantas Mikulėnas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/systemd-devel/attachments/20190925/ec454926/attachment.html>