[systemd-devel] Should `MACAddressPolicy=persistent` for bridges/bonds/all-software-devices be reconsidered?

[Dan Streetman]

On Thu, May 12, 2022 at 11:11 AM Thomas Haller <thaller at redhat.com> wrote:
>
> Hi Zbyszek,
>
>
>
> I must say, I personally don't care too much. NetworkManager is fine
> either way.
>
> There is however the problem about RHEL8/9, which patches this
> downstream. I don't like that deviation, but I'd also be uncomfortable
> to push that change for RHEL(10) users.
>
> But let me play devil's advocate here...
>
>
> On Mon, 2022-05-09 at 19:27 +0200, Zbigniew Jędrzejewski-Szmek wrote:
> >
> > FWIW, I still think it's a better _default_.
>
> I don't agree that it's clearly better. Your arguments don't seem
> strong, arguably, neither are mine. Except, that it's not clear that
> this solves an actual problem, while it clearly causes problems for
> some people. Just look at the referened issues from !3374.
>
>
> Either
>
>   - a user doesn't care about the MAC address,

note that it's possible for a user not to care about the *specific*
mac address, only that they want the mac to remain consistent.

for example, on my system i have br0 bridge with multiple interfaces
attached, and my local DHCP server is configured to provide a static
addr to br0's mac. If I replace an interface card (e.g. change from 1g
to 10g, or just replace a failing nic) then the br0 mac *does not*
change, if using systemd's default. If I had br0 inheriting its mac
from one of the attached interfaces, it would change, and i'd have to
update my dhcp server config.

I think the argument really comes down to, should the default be to
have (without user configuration) a mac that is predictable or a mac
that is consistent. Your argument is for predictability, which makes
the field engineer's work deploying systems easier, but if you lose
consistency then the life of the maintenance engineer gets harder.


>   - a user sets the desired MAC address themselves (possibly by
>     setting MacAddressPolicy=)
>   - or, the user intentionally does not set a MAC address (e.g. for
>     bridge/bond).
>
> In no case there is a strong argument why udev should do it, and it's
> harmful in some cases.
>
>
>
>
> >  The patch that finally
> > introduced this was my patch [1], so I'm obviously biased… Some more
> > considerations:
> >
> > 1. this allows bridge devices to be created without attached
> > interfaces, and have a stable MAC address.
>
> yes. Does somebody care? In particular, kernel will assign a certain
> MAC address when the first port gets attached. Some people care about
> that.
>
>
> > 2. the idea that all interfaces are always available and always in
> > the
> > same order is something that isn't necessarilly true for modern
> > systems
> > where we want to react to hardware being detected dynamically.
> > If we wait until late userspace to configure networking, then yeah,
> > all
> > devices will most likely have been detected by that time. But if we
> > want
> > to bring networking in the initrd, not all hardware must be detected
> > by
> > the time we start configuration.
>
> It's either way a well-known MAC address from one of the interfaces.
>
> >
> > 3. one of the reasons to use bridge/bond and similar devices it that
> > the agreggate device can function when some of the child devices die.
> > By requiring the presence of one of the devices, we're partially
> > defeating this.
>
> In practice, this race is not happening for many users.
>
> It's not clear why it's a problem to get the MAC address of an
> "unexpected" port (due to the race). It does not negate the uses of the
> bride/bond.
>
> >
> > [1] https://github.com/systemd/systemd/commit/6d36464065
> >
> > > 1) for bridge/bond interfaces, there is a special meaning of
> > > leaving
> > > the MAC address unassigned. It causes kernel to automatically set
> > > the
> > > MAC address when the first port gets attached. By setting a
> > > persistent
> > > MAC address, that automatism is not longer possible.
> > >
> > > The MAC address can matter, for example, if you configure the DHCP
> > > server to hand out IP addresses based on the MAC address (or based
> > > on
> > > the client-id, which in turn might be based on the MAC address). If
> > > you
> > > boot many machines (e.g. in data center or cloud), you might know
> > > the
> > > MAC address of the machines, and thereby can also determine the
> > > assigned IP addresses. With MACAddressPolicy=persistent the MAC
> > > address
> > > is not predictable.
> >
> > This is true. But one can just as well argument that with
> > MACAddressPolicy=persistent the address is even more predictable. If
> > you know the machine-id and device name, you can calculate the
> > address
> > in advance, even before deciding if the device will e.g. have this or
> > that card attached.
>
> It's unpredictable, because you need to know the machine-id. If you
> boot a machine the first, you may not know that before hand.
>
> Unless you explicitly specifying the machine-id via kernel command
> line, or similar. At which point, you need individual boot options (or
> images) for each machine.
>
> >
> > I'm not sure if we expose this conveniently anywhere… Would it help
> > if
> > we document how to do this calculation with python one-liner or some
> > helper?
> >
> > > 2) udev changing the MAC address causes races with naive
> > > scripts/tools.
> > > For example:
> > >
> > >   ip monitor link &
> > >   while : ; do
> > >     ip link del xxx
> > >     ip link add name xxx type dummy \
> > >     && ip link set xxx addr aa:00:00:00:00:00 \
> > >     && ip link show xxx | grep -q aa:00:00:00:00:00 \
> > >     || break
> > >   done
> >
> > Again, this is a question of expecations. With MACAP=p, 'dummy' gets
> > the same address every time, and maybe there's no reason to set it
> > manually.
> >
> > It would be great if we could have 'ip link add' wait for udev to
> > process the device. Maybe even by default?
>
>
>
> >
> > We also discussed adding a kernel command-line switch similar to
> > net.ifnames= to allow this to be configured globally. Would that
> > help?
>
> That might help. Granted, it's pretty easy already. RHEL ships a one-
> line patch downstream, and CoreOS probably could deploy some .link file
> that overrules this. And any user can write their own .link file.
>
> >
> > The cases where the old behaviour is relied on seems to be cases like
> > the data center described above. But in that case you're creating
> > local configuration anyway, so dropping in an override should be
> > acceptable.
>
> True. Except, this cuts both ways. Anybody who wants to enable this,
> could already do so.
>
> We are talking here about software device which are always created by
> some tool/software/user. Presumably that creator has plans for this
> interface, and it's not clear why udev is supposed to change such a
> fundamental setting.
>
>
>
>
> Thomas
>