[RFC] initoverlayfs - a scalable initial filesystem

[Eric Curtin]

On Sat, 9 Dec 2023 at 18:12, Luca Boccassi <bluca at debian.org> wrote:
>
> On Sat, 9 Dec 2023 at 17:58, Eric Curtin <ecurtin at redhat.com> wrote:
> >
> > On Sat, 9 Dec 2023 at 17:46, Luca Boccassi <bluca at debian.org> wrote:
> > >
> > > On Sat, 9 Dec 2023 at 17:25, Eric Curtin <ecurtin at redhat.com> wrote:
> > > >
> > > > On Sat, 9 Dec 2023 at 17:19, Luca Boccassi <bluca at debian.org> wrote:
> > > > >
> > > > > On Sat, 9 Dec 2023 at 15:08, Eric Curtin <ecurtin at redhat.com> wrote:
> > > > > >
> > > > > > On Sat, 9 Dec 2023 at 14:56, Andrei Borzenkov <arvidjaar at gmail.com> wrote:
> > > > > > >
> > > > > > > On 09.12.2023 17:42, Eric Curtin wrote:
> > > > > > > > On Sat, 9 Dec 2023 at 12:46, Luca Boccassi <bluca at debian.org> wrote:
> > > > > > > >>
> > > > > > > >> On Fri, 8 Dec 2023 at 19:00, Eric Curtin <ecurtin at redhat.com> wrote:
> > > > > > > >>>
> > > > > > > >>> We have been working on a new initial filesystem called initoverlayfs.
> > > > > > > >>> It is a new filesystem that provides a more scalable approach to
> > > > > > > >>> initial filesystems as opposed to just using initrds. We are writing
> > > > > > > >>> this RFC to the systemd and dracut mailing lists (feel free to forward
> > > > > > > >>> to UAPI group also) because although this solution works without
> > > > > > > >>> changing the code in these projects, it operates in the same area as
> > > > > > > >>> systemd, udev, dracut, etc. and uses these tools.
> > > > > > > >>
> > > > > > > >> It seems to me everything you described already exists? If you want to
> > > > > > > >> avoid having an initrd -> rootfs transition, you can already do that -
> > > > > > > >
> > > > > > > > You need a initrd -> rootfs transition for generic linux operating
> > > > > > > > systems right?
> > > > > > >
> > > > > > > No, you do not. Nothing stops you from running off initramfs (today you
> > > > > > > do not really have init*RAM Disk* - the content of initrd is unpacked
> > > > > > > into initramfs.
> > > > > >
> > > > > > Apologies if I am misinterpreting this response, I use terms initrd
> > > > > > and initramfs
> > > > > > interchangeably (not technically correct, but it's common to do this). The
> > > > > > point is to avoid unpacking as much as possible, because in many initrds
> > > > > > the majority of the software need not be unpacked, but is designed to work
> > > > > > with throwaway initial filesystems.
> > > > >
> > > > > sd-stub already supports having a small initrd shipped in the UKI,
> > > > > that is extended via sysexts, and systemd already supports running
> > > > > from it, without any transition to a final rootfs. What else do you
> > > > > need? What problem is this attempting to solve?
> > > >
> > > > I must give sd-stub a try. The bootloader I most commonly work with (and is one
> > > > of the target platforms this is intended for) isn't UEFI, we need something more
> > > > portable.
> > >
> > > Do we, though? All modern hardware platforms (and VMs) that matter are
> > > UEFI. Why would any of this be needed for legacy hardware platforms?
> > > The existing mechanisms can work just fine on those until they reach
> > > EOL, they won't stop working.
> >
> > Respectfully, this is not true. Especially on ARM platforms. I would
> > like it to be true, but it's not true today.
>
> Where any of this would actually matter, they mostly do, and where
> they don't one can put together uboot with uefi mode.

When you are trying to improve boot performance, introducing another
layer of bootloader with uboot doesn't help. You also have to port
every hardware platform you encounter to uboot. And if you can solve
the problem in the Linux stack somewhere rather than the bootloader.
Why would we choose to fix the problem in the bootloader?

>
> > I should have expanded, we are not trying to avoid transitioning to a
> > final rootfs, the goal is to transition to a final rootfs. But not to decompress
> > and copy all the bytes to a tmpfs up front, rather use something like erofs,
> > overlayfs, etc. sysexts uses erofs+overlayfs, but it's designed with
> > a different goal in mind.
>
> In what way is the goal different?

This project is basically build an initrd, but put it in a
erofs+overlayfs alternatively (technically it builds a really small
initrd to initialize some basic storage drivers etc. and build a
second initrd in an erofs format). All existing software that we've
tested "just works" with this approach, including all the systemd
stuff. And you can do transparent decompression with lz4hc
alternatively. It also means you don't have to be as afraid of
bloating your initial filesystem, because minimizing initrd's is
tedious work.

>