[systemd-devel] more verbose debug info than systemd.log_level=debug?

[Chris Murphy]

On Fri, May 19, 2017 at 1:46 PM, Chris Murphy <lists at colorremedies.com> wrote:
> FYI the file system folks are discussing this. It is not just a
> problem with XFS it can affect ext4 too. And it's far from clear the
> fs folks have a solution that won't cause worse problems.


OK so this is what I got out of those conversations

sync() -> write data to disk, write metadata to log
FIFREEZE() -> sync() and write log contents to fs.
unmount() -> sync() write log contents to fs.
reboot() -> sync() and reboot.

Only on non-journaled file systems does sync() mean write data to
disk, write metadata to fs, because there's no log.

sync() only makes the file system crash safe. It's doesn't mean the
bootloader can find files: configuration, kernel, initramfs, if they
are only sync()'d because the bootloader has no idea how to read the
log. And the fs itself isn't up to date because the log is dirty.

The most central blame here goes to the bootloader: specifically that
which makes changes to the bootloader configuration in a manner that
(pretty much) guarantees the bootloader proper (the binary that
executes after POST) will not be able to find either the old or new
configuration. At the least if it found the old configuration, it
would boot the old kernel and initramfs, which would then cause the
journal to be replayed, the file system updated, and on next boot, the
new configuration, kernel, and initramfs would get used. Because the
bootloader has a special requirement, since it cannot read dirty logs,
the thing making bootloader related changes needs to make sure that
its updates are not merely crash safe, but are actually fully
committed to the file system. That requires fsfreeze.

That implicates grub-mkconfig (for GRUB), grubby (not related to GRUB,
used on Red Hat, CentOS, Fedora systems), and myriad kernel package
scripts that modify bootloader configurations, kernels, and initramfs
out in the wild. The first two: grub-mkconfig and grubby, probably
represent a fairly good chunk of deployments. But there's still a
bunch of non-Red Hat systems that do not use GRUB, and do not use
grubby, they depend on the kernel package post-install scripts to make
bootloader changes, and that is what would need to do fsfreeze.

Or systemd can help pick up some of the slack and figure out a way to
either make sure one of three things definitely happens before a
reboot: umount, remount-ro, or fsfreeze. Of course, not every distro
uses systemd, and so only solving the central problem is a solution on
those distros, but in either case that's not systemd's responsibility.


-- 
Chris Murphy