[systemd-devel] Merging journal logs from btrfs snapshots

[Kai Krakow]

Chris Murphy <lists at colorremedies.com> schrieb:

>>> Due to anti-magic, a recent update horribly broke the system's ability
>>> to do further updates. This is resolved by regression to a prior Btrfs
>>> snapshot, once updated it works fine. But that's a two week old
>>> snapshot. I don't need the broken rootfs but I want to keep the journal
>>> for those two weeks.
>>> 
>>> Is this a reasonable want or need and if so how to merge the logs?
>>> Between the two snapshots there are several like named files in
>>> /var/log/journal/<machine-id>.
>> 
>> I'd recommend to place /var/log/journal on a subvolume so it is not
>> affected by snapshotting. You can do separate snapshots for it (tho I
>> cannot imagine why you would want to do it). That way you get a snapshot
>> "protection" for these files, too, and you are free to roll back the rest
>> of the system without affecting this subvolume.
> 
> Aha, good idea. So then I mount the subvol at /var/log/journal? Is there
> any risk of journald writing to rootfs /var/log/journal before the
> subvolume is mounted? Or is the flush to persistent storage sufficiently
> delayed as to not be a concern?

I guess systemd is intelligent enough to handle that case as it auto-creates 
dependecies for filesystem mounts. But if you want to be safe, specify your 
mount point with x-system.automount option is fstab. This would buffer 
access to the directory until it has been mounted and is ready to use.

OTOH, I'm not sure if this could create a deadlock during boot as journald 
is one of the essential services restarted as early as possible. I'm sure an 
expert here can give more details.

Using "systemctl show systemd-journald.service" and "systemctl show systemd-
journal-flush.service" should give you an idea how systemd would handle it.

I have not tried it yet but you gave me an interesting idea... ;-)

Regards,
Kai