[systemd-devel] systemd-networkd deletes local IPv6 routes for /128 addresses in VRF tables for VRF enslaved interfaces

[Marcel Menzel]

Hello list,

I am using a VRF with multiple Wireguard interfaces in it, and it
contains one dummy interface with a /128 IPv6 and a /32 IPv4 on it, all
managed by systemd-networkd.
This works until I restart systemd-networkd via systemctl restart
systemd-networkd, afterwards I am not able to ping the /128 IPv6 anymore.

The test setup to reproduce the behavior:

01-test-vrf.netdev:
    [NetDev]
    Name=test-vrf
    Kind=vrf

    [VRF]
    TableId=10

01-test-vrf.network:
    [Match]
    Name=test-vrf

    [Route]
    Destination=0.0.0.0/0
    Table=10
    Type=unreachable
    Metric=4278198272

    [Route]
    Destination=::/0
    Table=10
    Type=unreachable
    Metric=4278198272

02-test-dummy.netdev:
    [NetDev]
    Name=test-dummy
    Kind=dummy

02-test-dummy.network:
    [Match]
    Name=test-dummy

    [Network]
    VRF=test-vrf
    Address=fdde:11:22::1/128
    Address=fdde:33:44::1/64
    Address=10.20.30.1/32
    Address=10.20.40.1/24

Upon boot, everything works normally. I am able to ping all IPs on the
dummy interface:
    # ip vrf exec test-vrf ping fdde:11:22::1
    PING fdde:11:22::1(fdde:11:22::1) 56 data bytes
    64 bytes from fdde:11:22::1: icmp_seq=1 ttl=64 time=0.042 ms

    # ip vrf exec test-vrf ping fdde:33:44::1
    PING fdde:33:44::1(fdde:33:44::1) 56 data bytes
    64 bytes from fdde:33:44::1: icmp_seq=1 ttl=64 time=0.042 ms

    # ip vrf exec test-vrf ping 10.20.30.1
    PING 10.20.30.1 (10.20.30.1) 56(84) bytes of data.
    64 bytes from 10.20.30.1: icmp_seq=1 ttl=64 time=0.033 ms

    # ip vrf exec test-vrf ping 10.20.40.1
    PING 10.20.40.1 (10.20.40.1) 56(84) bytes of data.
    64 bytes from 10.20.40.1: icmp_seq=1 ttl=64 time=0.023 ms

And also the local routes have been moved to the VRF table aswell:
    # ip -6 r sh t 10 | grep local
    local fdde:11:22::1 dev test-dummy proto kernel metric 0 pref medium
    local fdde:33:44::1 dev test-dummy proto kernel metric 0 pref medium

    # ip r sh t 10 | grep local
    local 10.20.30.1 dev test-dummy proto kernel scope host src 10.20.30.1
    local 10.20.40.1 dev test-dummy proto kernel scope host src 10.20.40.1

But when I restart systemd-networkd (systemctl restart
systemd-networkd), the local route for the /128 IPv6 on the dummy
interface in the VRF table vanished:
    # ip r sh t 10 | grep local
    local 10.20.30.1 dev test-dummy proto kernel scope host src 10.20.30.1
    local 10.20.40.1 dev test-dummy proto kernel scope host src 10.20.40.1

    # ip -6 r sh t 10 | grep local
    local fdde:33:44::1 dev test-dummy proto kernel metric 0 pref medium

I am able to ping all addresses on the dummy interface except the /128
IPv6 one:
    # ip vrf exec test-vrf ping fdde:11:22::1
    PING fdde:11:22::1(fdde:11:22::1) 56 data bytes
    ^C
    --- fdde:11:22::1 ping statistics ---
    2 packets transmitted, 0 received, 100% packet loss, time 1000ms

To fix this, I either have to delete the test-dummy interface and
restart systemd-networkd afterwards (ip l del test-dummy && systemctl
restart systemd-networkd),
or add this local route by hand again: ip -6 r add local fdde:11:22::1
dev test-dummy proto kernel metric 0 pref medium table 10

The metric gets changed to 1024 though so I have to delete the non-local
route for this interface in order to be able to ping it again so I
prefer the first approach:
    # ip vrf exec test-vrf ping fdde:11:22::1
    PING fdde:11:22::1(fdde:11:22::1) 56 data bytes
    ^C
    --- fdde:11:22::1 ping statistics ---
    1 packets transmitted, 0 received, 100% packet loss, time 0ms

    # ip -6 r sh t 10
    fdde:11:22::1 dev test-dummy proto kernel metric 256 pref medium
    local fdde:11:22::1 dev test-dummy proto kernel metric 1024 pref medium

    # ip -6 r del fdde:11:22::1 dev test-dummy table 10

    # ip vrf exec test-vrf ping fdde:11:22::1
    PING fdde:11:22::1(fdde:11:22::1) 56 data bytes
    64 bytes from fdde:11:22::1: icmp_seq=1 ttl=64 time=0.038 ms

I was able to reproduce this behavior with a Wireguard interface aswell,
so I think this behavior applies to all netdev types. Also worth to
mention, "networkctl reload" does not trigger this behavior aswell.

The kernel docs say:

   Local and connected routes for enslaved devices are automatically
moved to
   the table associated with VRF device. Any additional routes depending on
   the enslaved device are dropped and will need to be reinserted to the VRF
   FIB table following the enslavement.

When commenting out the "VRF=test-vrf" on the dummy's .network file and
enslaving it by hand (ip l set dev test-dummy master test-vrf), this
works aswell as expected until I restart systemd-networkd and I have to
enslave it again and do above steps.

Am I missing something out or did I hit a bug here? Version is systemd
245 (245.5-2-arch) on 5.6.13-arch1-1 (Arch Linux).

Kind regards,

Marcel Menzel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/systemd-devel/attachments/20200517/174a4aec/attachment.htm>