[systemd-devel] [PATCH] firmware: wake all waiters

[Luis R. Rodriguez]

On Wed, Jun 28, 2017 at 06:21:09PM +0200, Lennart Poettering wrote:
> On Wed, 28.06.17 09:06, Luis R. Rodriguez (mcgrof at kernel.org) wrote:
> 
> > On Wed, Jun 28, 2017 at 12:06 AM, Lennart Poettering
> > <mzxreary at 0pointer.de> wrote:
> > > On Wed, 28.06.17 00:24, Luis R. Rodriguez (mcgrof at kernel.org) wrote:
> > >
> > >> > Do you know how systemd developers feel about the issue (CCed)?  Given
> > >> > that it seems to dominate in data center OSes now I'm slightly worried
> > >> > having to push Big Linux Vendors to package some seemingly
> > >> > embedded-centric software just to make advanced NICs run :(
> > >>
> > >> firmwared was written by a systemd developer :)
> > >
> > > No it wasn't. I don't know what firmwared is really. Sorry.
> > 
> > Is Tom Gundersen not a systemd developer?
> 
> Not really anymore, and "firmwared" is an effort independent of
> systemd, never was part of it, and while I heard Tom was working on
> this I was not aware of the project's naming or anything else...

Alright, thanks for the clarifications and sorry for the confusion!

In that case firmwared remains *just* an architecture example of an alternative
to the problem of looking for firmware through a *fallback mechanism* and
addressing "is my real rootfs mounted yet" problem some folks have struggled to
resolve, "are we sure we're ready to look for all firmware?".

Lennart, if you have a better architectural suggestion let us know.

I realize that the firmware fallback mechanism was ripped out of systemd long
ago, specifically as of systemd commit be2ea723b1d0 (“udev: remove userspace
firmware loading support”) as of v217 on August, 2014. This means most Linux
distributions today are not using or taking advantage of the firmware fallback
mechanism provided by kobject uevents. 

This is specially exacerbated due to the fact that most distributions today
disable CONFIG_FW_LOADER_USER_HELPER_FALLBACK which *means* only the custom
fallback mechanism (no uevents are issued) can be used on those distributions,
and we actually want to *avoid* having more drivers use that mechanism. Only
2 drivers remain upstream now which explicitly require the custom fallback
mechanism. We don't to add any more.

This leaves distributions that want a fallback mechanism today only with the
option to enable CONFIG_FW_LOADER_USER_HELPER_FALLBACK and rely on uevents, and
firmwared was an architectural example of how to address the rootfs problem.

Android is enabling CONFIG_FW_LOADER_USER_HELPER_FALLBACK these days it seems.

If it helps the fallback mechanism is now documented here:

https://www.kernel.org/doc/html/latest/driver-api/firmware/fallback-mechanisms.html

The concept of firmwared was simple: it had best-effort mode and final-mode.
It relies on CONFIG_FW_LOADER_USER_HELPER_FALLBACK=y and relies on uevents.
You boot with it on best-effort mode where firmware is hunted for in a best
effort way, but it does not fail a load through the kernel's sysfs interface
used for the fallback mechanism. Then once userspace knows we have reached the
real rootfs (since only it knows when this happens) it kicks firmwared into
final-mode, which in turn can now iterate over pending firmware and send a "not
found" with certainty.

So the focus for now is ironing out something that we know works *very well*
for the CONFIG_FW_LOADER_USER_HELPER_FALLBACK=y folks.

It'd be great if we had a solution that could work for
CONFIG_FW_LOADER_USER_HELPER_FALLBACK=n folks but its unclear if that's
possible, so it may be best to only revisit this if and when we know for sure
CONFIG_FW_LOADER_USER_HELPER_FALLBACK=y and the rootfs issue is properly ironed
out with the uevents fallback mechanism.

  Luis