[systemd-devel] journal fragmentation on Btrfs

[Lennart Poettering]

On Mon, 17.04.17 21:50, Chris Murphy (lists at colorremedies.com) wrote:

> >> Why do I see so many changes to the journal file, once ever 2-5
> >> seconds? This adds 4096 byte blocks to the file each time, and when
> >> cow, that'd explain why there are so many fragments.
> >>
> >
> >
> > What exactly "changes" mean? Write() syscall?
> 
> filefrag reported entries increase, it's using FIEMAP.

As mentioned we write to the file via mmap() as we recv the log
messages, and then issue ftruncate()'s to propagate mtime inotify
events which other clients can watch for for live log views. And then,
5min after a write we issue sync(), but at most every 5min once.

> Also with stat I see the times (all three) change on the file. If I go
> to GNOME Terminal and just sudo some command, that itself causes the
> current system.journal file to get all three times modified. It
> happens immediately, there's no delay. So if I'm doing something like
> drm.debug=0x1e, which is spitting a bunch of stuff to dmesg and thus
> the journal, it's just constantly writing stuff to the journal. This
> is without anything running journalctl -f or reading the journal.

The "sudo" command logs each invocation, hence, yes, of course, the
log files will get updated.

> >
> >> #Storage=auto
> >> #Compress=yes
> >> #Seal=yes
> >> #SplitMode=uid
> >> #SyncIntervalSec=5m
> >
> > This controls how often systemd calls fsync() on currently active
> > journal file. Do you see fsync() every 3 seconds?
> 
> I have no idea if it's fsync or what. How can I tell?

You can do "strace -p `pidof systemd-journald` -e sync"...

> Also, I don't think these journal files are being compressed.
> 
> Using the btrfs-progs/btrfs-debugfs script on a few user journal
> files, I'm seeing massive compression ratios. Maybe I'll try
> Compress=No and see if there's a change.

As documented Compress= will only compress large objects stored in the
journal, but not the general journal structure. This means journal
files are usually highly compressible, still. Random access and
compression don't easily mix, and we valued random access more.

Lennart

-- 
Lennart Poettering, Red Hat