[systemd-devel] systemd prerelease 247-rc2

[Zbigniew Jędrzejewski-Szmek]

On Thu, Nov 12, 2020 at 04:36:01PM +0100, Michael Biebl wrote:
> Hi there
> 
> Am Do., 12. Nov. 2020 um 11:58 Uhr schrieb systemd tag bot
> <donotreply-systemd-tag at refi64.com>:
> >
> > A new systemd ☠️ pre-release ☠️ has just been tagged. Please download the tarball here:
> >
> >         https://github.com/systemd/systemd/archive/v247-rc2.tar.gz
> 
> Congrats to the new release!
> 
> > Changes since the previous release:
> >
> >         * KERNEL API INCOMPATIBILITY: Linux 4.12 introduced two new uevents
> >           "bind" and "unbind" to the Linux device model. When this kernel
> >           change was made, systemd-udevd was only minimally updated to handle
> >           and propagate these new event types. The introduction of these new
> >           uevents (which are typically generated for USB devices and devices
> >           needing a firmware upload before being functional) resulted in a
> >           number of issues which we so far didn't address. We hoped the kernel
> >           maintainers would themselves address these issues in some form, but
> >           that did not happen. To handle them properly, many (if not most) udev
> >           rules files shipped in various packages need updating, and so do many
> >           programs that monitor or enumerate devices with libudev or sd-device,
> >           or otherwise process uevents. Please note that this incompatibility
> >           is not fault of systemd or udev, but caused by an incompatible kernel
> >           change that happened back in Linux 4.12, but is becoming more and
> >           more visible as the new uevents are generated by more kernel drivers.
> >
> >           To minimize issues resulting from this kernel change (but not avoid
> >           them entirely) starting with systemd-udevd 247 the udev "tags"
> >           concept (which is a concept for marking and filtering devices during
> >           enumeration and monitoring) has been reworked: udev tags are now
> >           "sticky", meaning that once a tag is assigned to a device it will not
> >           be removed from the device again until the device itself is removed
> >           (i.e. unplugged). This makes sure that any application monitoring
> >           devices that match a specific tag is guaranteed to both see uevents
> >           where the device starts being relevant, and those where it stops
> >           being relevant (the latter now regularly happening due to the new
> >           "unbind" uevent type). The udev tags concept is hence now a concept
> >           tied to a *device* instead of a device *event* — unlike for example
> >           udev properties whose lifecycle (as before) is generally tied to a
> >           device event, meaning that the previously determined properties are
> >           forgotten whenever a new uevent is processed.
> >
> >           With the newly redefined udev tags concept, sometimes it's necessary
> >           to determine which tags are the ones applied by the most recent
> >           uevent/database update, in order to discern them from those
> >           originating from earlier uevents/database updates of the same
> >           device. To accommodate for this a new automatic property CURRENT_TAGS
> >           has been added that works similar to the existing TAGS property but
> >           only lists tags set by the most recent uevent/database
> >           update. Similarly, the libudev/sd-device API has been updated with
> >           new functions to enumerate these 'current' tags, in addition to the
> >           existing APIs that now enumerate the 'sticky' ones.
> >
> >           To properly handle "bind"/"unbind" on Linux 4.12 and newer it is
> >           essential that all udev rules files and applications are updated to
> >           handle the new events. Specifically:
> >
> >           • All rule files that currently use a header guard similar to
> >             ACTION!="add|change",GOTO="xyz_end" should be updated to use
> >             ACTION=="remove",GOTO="xyz_end" instead, so that the
> >             properties/tags they add are also applied whenever "bind" (or
> >             "unbind") is seen. (This is most important for all physical device
> >             types — those for which "bind" and "unbind" are currently
> >             generated, for all other device types this change is still
> >             recommended but not as important — but certainly prepares for
> >             future kernel uevent type additions).
> >
> >           • Similarly, all code monitoring devices that contains an 'if' branch
> >             discerning the "add" + "change" uevent actions from all other
> >             uevents actions (i.e. considering devices only relevant after "add"
> >             or "change", and irrelevant on all other events) should be reworked
> >             to instead negatively check for "remove" only (i.e. considering
> >             devices relevant after all event types, except for "remove", which
> >             invalidates the device). Note that this also means that devices
> >             should be considered relevant on "unbind", even though conceptually
> >             this — in some form — invalidates the device. Since the precise
> >             effect of "unbind" is not generically defined, devices should be
> >             considered relevant even after "unbind", however I/O errors
> >             accessing the device should then be handled gracefully.
> >
> >           • Any code that uses device tags for deciding whether a device is
> >             relevant or not most likely needs to be updated to use the new
> >             udev_device_has_current_tag() API (or sd_device_has_current_tag()
> >             in case sd-device is used), to check whether the tag is set at the
> >             moment an uevent is seen (as opposed to the existing
> >             udev_device_has_tag() API which checks if the tag ever existed on
> >             the device, following the API concept redefinition explained
> >             above).
> >
> >           We are very sorry for this breakage and the requirement to update
> >           packages using these interfaces. We'd again like to underline that
> >           this is not caused by systemd/udev changes, but result of a kernel
> >           behaviour change.
> 
> I have to admit, it's unclear to me, what exactly this change means.
> As far as udev_device_has_tag() is concerned, it seems the Debian
> archive only ships one package (plymouth) which makes use of it.
> https://codesearch.debian.net/search?q=udev_device_has_tag+package%3A%5CQplymouth%5CE&literal=1
> 
> But what about all those packages shipping udev rules files. Does this
> new concept of sticky device tags affect them?
> And if so, in what way?
> 
> I mean, Linux kernel 4.2 was already released a while ago (with the
> new bind/unbind events). I assume most packages have been updated in
> the meantime to handle those accordingly.

Not necessarily: apart from the framework change in 4.2, individual
drivers needed to be updated for this to be visible to user space. And
various drivers were updated in later kernel versions.

Some packages have been updated, while others not. For example:

sg3_utils-1.45-3.fc33.x86_64 has: ACTION!="add|change", GOTO="usb_blacklist_end"
I'm pretty sure that add|bind|change is appropriate here, at least
for some devices. But answering what the effect of lack of "bind"
in this rule is is a hard question. It depends on the details of
hardware and drivers.

I have a bunch of other rules with ACTION!="add|change" on my machine.
It seems that if a bind event occurs, those devices will at least
not have properly assigned metadata. Probably not the end of the
world, but still something noticeable for users.

> What exactly changes with those new sticky udev tags?
> How can I find udev rules / packages which are potentially affected?
> Has this new concept the potential to break (many) existing udev rules?
> I'm now unsure whether it's safe to upload 247 to Debian unstable or not.

The tags change is probably limited in effect, and the lack of handling
of "bind" in rules will be more important. But, as you said, that change
was already happening since kernel 4.2. So it's possible that the biggest
effect is that with our announcement more people will notice that things
are not working as expected. I think it's safe to push 247 to users, but
be prepared for some bug reports.

Zbyszek