[systemd-devel] systemd-nspawn: cannot join existing macvlan

[Kai Krakow]

Lennart Poettering <lennart at poettering.net> schrieb:

> On Mon, 25.05.15 16:26, Kai Krakow (hurikhan77 at gmail.com) wrote:
> 
>> Lennart Poettering <lennart at poettering.net> schrieb:
>> 
>> > On Fri, 08.05.15 20:53, Kai Krakow (hurikhan77 at gmail.com) wrote:
>> > 
>> >> > # systemd-nspawn -b --link-journal=try-guest
>> >> > # --network-macvlan=enp4s0 --
>> >> > bind=/usr/portage --bind-ro=/usr/src --machine=test
>> >> > Spawning container test on /var/lib/machines/test.
>> >> > Press ^] three times within 1s to kill container.
>> >> > Failed to add new macvlan interfaces: File exists
>> >> > 
>> >> > I still don't think that systemd-nspawn should insist on creating
>> >> > the host- side macvlan bridge and fail, if it cannot. It should just
>> >> > accept that it is already there.
>> >> 
>> >> My findings show that it actually does accept this case. But I had to
>> >> explicitly order the machines after network.target to successfully
>> >> start at boot time.
>> > 
>> > I changed git now to order nspawn units by default after
>> > network.target now. In the long run we should replace this by calling
>> > into networkd though, without waiting for "all" networking, for
>> > whatever that means...
>> 
>> Well, now, with v220, we are back to the original problem:
>> 
>> machines # systemd-nspawn --boot --link-journal=try-guest --machine=test
>> -- network-macvlan=enp5s0
>> Spawning container test on /var/lib/machines/test.
>> Press ^] three times within 1s to kill container.
>> Failed to add new macvlan interfaces: File exists
>> 
>> If mv-enp5s0 is already there (maybe by means of another machine), it no
>> longer starts any other machine on the same macvlan. I don't think this
>> is how it should work.
> 
> Well, there can only be one machine with the same name, and we use
> that name in the macvlan interface name. Please assign different names
> to your containers so that they will also get differently named
> macvlan names, and all should be good.

But what is the purpose of macvlan if you cannot merge different containers 
into the same LAN segment? One feature of macvlan is that all virtual MAC 
addresses can communicate with each other. That's why I also joined my host 
machine into that macvlan (by creating it with networkd).

The next issue with your argument is: AFAIR nspawn doesn't create a macvlan 
interface based on the machine name. You have to pass the name of a physical 
interface which transports this macvlan. The man page at least states that 
you use an existing physical interface:

    --network-macvlan=
        Create a "macvlan" interface of the specified Ethernet network
        interface and add it to the container. A "macvlan" interface is a
        virtual interface that adds a second MAC address to an existing
        physical Ethernet link. The interface in the container will be named
        after the interface on the host, prefixed with "mv-". Note that
        --network-macvlan= implies --private-network. This option may be
        used more than once to add multiple network interfaces to the
        container.

Trying to nspawn a macvlan without giving a physical device results in "no 
such device":

# systemd-nspawn --boot --link-journal=try-guest --machine=gentoo-mysql-base 
--network-macvlan=
systemd-nspawn: option '--network-macvlan' requires an argument

# systemd-nspawn --boot --link-journal=try-guest --machine=test --network-
macvlan=test
Spawning container test on /var/lib/machines/test.
Press ^] three times within 1s to kill container.
Failed to resolve interface test: No such device

So your assumption about macvlan seems to be incorrect. The other network 
types may be based off the machine name but it doesn't work this way with 
macvlan.

So I wonder what the purpose of macvlan is if you can only create one 
machine using macvlan on my single physical link - and that's it?

I think the logic is wrong here in systemd-nspawn. Instead of trying to 
create the host-side macvlan itself it should insist of it being there 
already (to have one well-defined state to start with, and only optionally 
create it by itself). Then, it can join multiple machines to the same 
macvlan.

I'm using the following networkd config on the host to define this "well-
defined state", enabling the host to also communicate through macvlan:

# cat /etc/systemd/network/10-macvlan.{netdev,network}
[NetDev]
Name=mv-enp5s0
Kind=macvlan

[MACVLAN]
Mode=bridge
#----------------
[Match]
Name=en*

[Network]
MACVLAN=mv-enp5s0
LinkLocalAddressing=no
DHCP=no

If I remove this I am able to start one of the containers, but not two or 
more. If I use this I am not able to start any macvlan container.

-- 
Replies to list only preferred.