systemd-sysupdate support for slow rollout (aka A/B testing)

[Lennart Poettering]

On Di, 02.01.24 13:11, Simon McVittie (smcv at collabora.com) wrote:

> Prior art: Debian/Ubuntu apt does slow rollout for packages like
> this, with simple filesystem-based http mirrors combined with "smart"
> clients. It works by adding a Phased-Update-Percentage field to the
> metadata of each package. The client calculates some sort of ID for itself
> (I don't know precisely how), and then takes the upgrade if it finds that
> its ID is in the first x% of the available range.
>
> If I understand correctly, Ubuntu is using this mechanism in production
> but Debian is not.
>
> Using some sort of hash of the machine ID + the proposed version would
> probably have the behaviour you want, of choosing a different x% of
> machines to be the early-adopter set for each update?

Yes, this is what I think would be the right approach.

> > This would then mean for the server that it would first serve
> > foobar_47.11_3.raw which would be version 47.11 of the OS, and 3% of
> > the hosts would update to it. And then, once you collected enough
> > feedback you'd rename the file to foobar_47.11_25.raw and 25% of the
> > hosts would switch over. Finally you'd set the value to 100 (or maybe
> > just drop it, which should be considered equivalent to 100), and then
> > all remaining hosts would update.
>
> If you're using a hash of the machine ID + the proposed version as
> your randomization, then I think you'd want to have a single image (or
> version ID, or some other unique identifier) for each proposed update, and
> separately, a metadata field that sets *x* in the instruction "if you have
> figured out that you are in the first x% of machines, upgrade". Otherwise,
> publishing foobar_47.11_3.raw followed by foobar_47.11_25.raw would be
> more likely to result in approximately (3% + 25% = 28%) of machines
> upgrading[1], because the client doesn't know that it's actually the
> same update and would "re-roll the dice" for each republished name.

My thinking was that clients would look at multiple entries which only
differ by the percentage (i.e. are identical in name and version) and
drop all of them but the one with the highest percentage, and ignore
all others.

Lennart

--
Lennart Poettering, Berlin