[systemd-devel] more verbose debug info than systemd.log_level=debug?

[Chris Murphy]

On Tue, Mar 28, 2017 at 10:41 AM, Mantas Mikulėnas <grawity at gmail.com> wrote:
> On Tue, Mar 28, 2017 at 5:01 PM, Chris Murphy <lists at colorremedies.com>
> wrote:
>>
>> On Mon, Mar 27, 2017 at 1:27 PM, Mantas Mikulėnas <grawity at gmail.com>
>> wrote:
>> > On Mon, Mar 27, 2017 at 10:20 PM, Chris Murphy <lists at colorremedies.com>
>> > wrote:
>> >>
>> >> Ok so the dirty file system problem always happens with all pk offline
>> >> updates on Fedora using either ext4 or XFS with any layout; and it's
>> >> easy to reproduce.
>> >>
>> >> 1. Clean install any version of Fedora, defaults.
>> >> 2. Once Gnome Software gives notification of updates, Restart & Install
>> >> 3. System reboots, updates are applied, system reboots again.
>> >> 4. Now check the journal filtering for 'fsck' and you'll see it
>> >> replayed the journals; if using XFS check the filter for "XFS" and
>> >> you'll see the kernel did journal replace at mount time.
>> >>
>> >> Basically systemd is rebooting even though the remoun-ro fails
>> >> multiple times, due to plymouth running off root fs and being exempt
>> >> from being killed, and then reboots anyway, leaving the file system
>> >> dirty. So it seems like a flaw to me to allow an indefinite exemption
>> >> from killing a process that's holding a volume rw, preventing
>> >> remount-ro before reboot.
>> >>
>> >> So there's a risk that in other configurations this makes either ext4
>> >> and XFS systems unbootable following an offline update.
>> >
>> >
>> > So on the one hand it's probably a Plymouth bug or misconfiguration (it
>> > shouldn't mark itself exempt unless it runs off an in-memory initramfs).
>>
>> OK. But does it even make sense to have a process exempt from killing,
>> when it's going to get killed by reboot? Seems to me once we're at
>> remount-ro or umount time, nothing is exempt, they need to be forcibly
>> killed, clean up the file system, and then reboot.
>
>
> Processes are killed *before* the remount/unmount stage. The primary users
> of kill-exemption would therefore be daemons which *provide* access to the
> root filesystem, e.g. iscsid, rpc helper daemons, or even ntfs-3g.
> (Naturally these are expected to be running from the initramfs.)

OK but it's obviously possible for a developer to run a process from
root fs, and mark it kill exempt. That's the problem under discussion,
the developer is doing the wrong thing, and it's allowed. And it's
been going on for a very long time (at least 5 releases of Fedora)


> So the same applies to plymouth, IMO -- it should only mark itself exempt if
> it runs from the initramfs and knows that it won't interfere.

How is this exemption specified? Would it be part of the plymouth packaging?

I recognize the immediate bug is with plymouth so to progress this
forward I'm happy to assign the Fedora bug and leave it up to Fedora
devs to figure out whether plymouth should go in the initramfs, or
remove the kill exemption. But long term, I still think there's a roll
for systemd here, to disregard process kill exemption if they're
running from a volume that's about to be remounted-ro or umounted.
Preventing that is asking for big problems, as seen here.



-- 
Chris Murphy