[systemd-devel] Trying to decipher a patch made in Centos 7 RPMs

[Robert Winslow Dalpe]

Thanks for providing input, Mantas. I appreciate the second opinion. I'll
try building an RPM without the patch (or at least with a new one
re-instating the LZ4 preprocessor directives) to confirm it has an impact
on the issue I'm investigating.

Robert Winslow Dalpe

On Tue, Sep 24, 2019 at 10:27 PM Mantas Mikulėnas <grawity at gmail.com> wrote:

> On Wed, Sep 25, 2019 at 6:32 AM Robert Winslow Dalpe <rwdalpe at gmail.com>
> wrote:
>
>> Hello,
>>
>> I'm trying to track down the reason for an issue I'm seeing between
>> systemd-journald and how rsyslog where data is being truncated when read
>> from the journal file ( https://github.com/rsyslog/rsyslog/issues/3784 ).
>>
>> The behavior seems present in systemd 219 (on amazon linux 2 and centos
>> 7) and not in later versions (on arch linux). I figured the truncation may
>> be due to the default threshold for sd_journal_get_data (
>> https://www.freedesktop.org/software/systemd/man/sd_journal_get_data.html
>> ). From what I can tell, it looks like the only time sd_journal_get_data
>> uses the threshold is when calling `decompress_blob` (
>> https://github.com/systemd/systemd/blob/a88abde72169ddc2df77df3fa5bed30725022253/src/journal/sd-journal.c#L1916
>> ). `decompress_blob` looks to see if the data is compressed using XZ or LZ4
>> and then decompresses appropriately (
>> https://github.com/systemd/systemd/blob/a88abde72169ddc2df77df3fa5bed30725022253/src/journal/compress.c#L214
>> ). It looks like only the XZ decompression code utilizes the threshold
>> (`dst_max`--which contains the threshold value--is unused in
>> `decompress_blob_lz4`).
>>
>> From what I can tell LZ4 compression is the default. Assuming I'm on the
>> right track, I'm trying to determine how XZ decompression would be invoked.
>> That led me to the RPM that centos7 (and thus presumably amazon linux 2)
>> builds for systemd. That RPM applies the following patch:
>> https://git.powerel.org/powerel7/base/src/commit/e9d94968812dc1b501e6c691ee8c01595d77f76d/SOURCES/0575-Add-support-to-read-lz4-compressed-journals.patch
>>  Although the patch says it "adds support to read lz4 compressed journals,"
>> it looks to me like it is actually making XZ compression the default by
>> removing several preprocessor directives that would allow for LZ4
>> compression/decompression. Am I interpreting that patch correctly? Does any
>> of what I've said seem even close to sane? I realize I can remove that
>> patch and try building an RPM myself, but I'm hoping someone might know
>> enough to provide any quick feedback before I go down that road.
>>
>
> Without the patch, XZ is already the default. The patch adds LZ4
> decompression support, but tries to avoid making LZ4 the *default* format
> for compression, making sure that e.g. compress_lz4 remains NULL despite
> the LZ4 library being present. (I'm guessing one reason is that the LZ4
> library already changed its framing format once, and CentOS wants to stick
> with the stable xz format?)
>
> --
> Mantas Mikulėnas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/systemd-devel/attachments/20191003/fc24db11/attachment.html>