[systemd-devel] Should `MACAddressPolicy=persistent` for bridges/bonds/all-software-devices be reconsidered?

[Dusty Mabe]

On 5/9/22 13:27, Zbigniew Jędrzejewski-Szmek wrote:
> On Mon, May 09, 2022 at 03:57:21PM +0200, Lennart Poettering wrote:
>> On Mo, 09.05.22 11:23, Thomas Haller (thaller at redhat.com) wrote:
>>
>>> Hi everybody,
>>>
>>> this email is for discussing MACAddressPolicy=persistent in
>>> /data/src/systemd/network/99-default.link
>>
>> I think this would be better discussed on a new github issue, as
>> suggested here:
> 
> I suggested systemd-devel for this… It's not a question of a bug,
> but instead whether the default MACAddressPolicy makes sense. So I think
> it's better to discuss this on the mailing list.
> 
> FWIW, I still think it's a better _default_. The patch that finally
> introduced this was my patch [1], so I'm obviously biased… Some more
> considerations:
> 
> 1. this allows bridge devices to be created without attached
> interfaces, and have a stable MAC address.
> 
> 2. the idea that all interfaces are always available and always in the
> same order is something that isn't necessarilly true for modern systems
> where we want to react to hardware being detected dynamically.
> If we wait until late userspace to configure networking, then yeah, all
> devices will most likely have been detected by that time. But if we want
> to bring networking in the initrd, not all hardware must be detected by
> the time we start configuration.
> 
> 3. one of the reasons to use bridge/bond and similar devices it that
> the agreggate device can function when some of the child devices die.
> By requiring the presence of one of the devices, we're partially
> defeating this.
> 
> [1] https://github.com/systemd/systemd/commit/6d36464065



> 
>> 1) for bridge/bond interfaces, there is a special meaning of leaving
>> the MAC address unassigned. It causes kernel to automatically set the
>> MAC address when the first port gets attached. By setting a persistent
>> MAC address, that automatism is not longer possible.
>>
>> The MAC address can matter, for example, if you configure the DHCP
>> server to hand out IP addresses based on the MAC address (or based on
>> the client-id, which in turn might be based on the MAC address). If you
>> boot many machines (e.g. in data center or cloud), you might know the
>> MAC address of the machines, and thereby can also determine the
>> assigned IP addresses. With MACAddressPolicy=persistent the MAC address
>> is not predictable.
> 
> This is true. But one can just as well argument that with
> MACAddressPolicy=persistent the address is even more predictable. If
> you know the machine-id and device name, you can calculate the address
> in advance, even before deciding if the device will e.g. have this or
> that card attached.


Regarding machine-id, isn't that unique and set on first boot? So you don't
really even know your "new mac address" until your system is up and running
(and now can't get DHCP because your net is locked down). 

For me it's more about expectations and the user being surprised, which isn't
necessarily a bug. If I'm controlling a datacenter (or even my 10s of devices
on my home network) I generally know what hardware exists in the environment
and don't want to be surprised to see activity from "unexpected hardware". I
lock down my DHCP via MAC address of the known hardware NICs in the lab/DC.

Often times hardware gets racked and configured in the datacenter environment
before it can even be powered on. You get the MAC address from the documentation
(labels, stickers, etc) and have your network admins plumb through the necessary
bits.

I understand your arguments for the new behavior, but my preference would be
to keep the old behavior. 


> 
> I'm not sure if we expose this conveniently anywhere… Would it help if
> we document how to do this calculation with python one-liner or some
> helper?
> 
>> 2) udev changing the MAC address causes races with naive scripts/tools.
>> For example:
>>
>>   ip monitor link & 
>>   while : ; do 
>>     ip link del xxx
>>     ip link add name xxx type dummy \
>>     && ip link set xxx addr aa:00:00:00:00:00 \
>>     && ip link show xxx | grep -q aa:00:00:00:00:00 \
>>     || break
>>   done
> 
> Again, this is a question of expecations. With MACAP=p, 'dummy' gets
> the same address every time, and maybe there's no reason to set it
> manually.
> 
> It would be great if we could have 'ip link add' wait for udev to
> process the device. Maybe even by default?
> 
> We also discussed adding a kernel command-line switch similar to
> net.ifnames= to allow this to be configured globally. Would that
> help?

Right. I opened that request here: https://github.com/systemd/systemd/issues/23294

I think it helps. I would want to make sure it applied in the initrd as well.

> 
> The cases where the old behaviour is relied on seems to be cases like
> the data center described above. But in that case you're creating
> local configuration anyway, so dropping in an override should be
> acceptable.

At least for bonds I'd argue pretty hard that the "data center described above"
is the primary place they get used. Bridges have other uses, but still highly
applicable if not the primary.

When you say "you're creating local configuration anyway", I think you're talking
about the configuration for the bond. That's a fair assessment but it can be
confusing to the user. At least in the case of NetworkManager I now configure
the bond in two places (NM and systemd) and also in the initramfs (if I'm using
initrd networking) or via kernel command line. It starts to get hairy. 

At the end of the day we're talking about the default here. It's fair to say
that a person "laying down configuration" could configure things either way.
The way I'm approaching the question of what should be the default is what is
least surprising to the user. I've made a case for what I think is less surprising,
but I'm interested in other thoughts on this one.

Dusty