<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<font size="+1">Hello list,<br>
<br>
I am using a VRF with multiple Wireguard interfaces in it, and it
contains one dummy interface with a /128 IPv6 and a /32 IPv4 on
it, all managed by systemd-networkd.<br>
This works until I restart systemd-networkd via systemctl restart
systemd-networkd, afterwards I am not able to ping the /128 IPv6
anymore.<br>
<br>
The test setup to reproduce the behavior:<br>
<br>
01-test-vrf.netdev:<br>
[NetDev]<br>
Name=test-vrf<br>
Kind=vrf<br>
<br>
[VRF]<br>
TableId=10<br>
<br>
01-test-vrf.network:<br>
[Match]<br>
Name=test-vrf<br>
<br>
[Route]<br>
Destination=0.0.0.0/0<br>
Table=10<br>
Type=unreachable<br>
Metric=4278198272<br>
<br>
[Route]<br>
Destination=::/0<br>
Table=10<br>
Type=unreachable<br>
Metric=4278198272<br>
<br>
02-test-dummy.netdev:<br>
[NetDev]<br>
Name=test-dummy<br>
Kind=dummy<br>
<br>
02-test-dummy.network:<br>
[Match]<br>
Name=test-dummy<br>
<br>
[Network]<br>
VRF=test-vrf<br>
Address=fdde:11:22::1/128<br>
Address=fdde:33:44::1/64<br>
Address=10.20.30.1/32<br>
Address=10.20.40.1/24<br>
<br>
Upon boot, everything works normally. I am able to ping all IPs on
the dummy interface:<br>
# ip vrf exec test-vrf ping fdde:11:22::1<br>
PING fdde:11:22::1(fdde:11:22::1) 56 data bytes<br>
64 bytes from fdde:11:22::1: icmp_seq=1 ttl=64 time=0.042 ms<br>
<br>
# ip vrf exec test-vrf ping fdde:33:44::1<br>
PING fdde:33:44::1(fdde:33:44::1) 56 data bytes<br>
64 bytes from fdde:33:44::1: icmp_seq=1 ttl=64 time=0.042 ms<br>
<br>
# ip vrf exec test-vrf ping 10.20.30.1<br>
PING 10.20.30.1 (10.20.30.1) 56(84) bytes of data.<br>
64 bytes from 10.20.30.1: icmp_seq=1 ttl=64 time=0.033 ms<br>
<br>
# ip vrf exec test-vrf ping 10.20.40.1<br>
PING 10.20.40.1 (10.20.40.1) 56(84) bytes of data.<br>
64 bytes from 10.20.40.1: icmp_seq=1 ttl=64 time=0.023 ms<br>
<br>
And also the local routes have been moved to the VRF table aswell:<br>
# ip -6 r sh t 10 | grep local<br>
local fdde:11:22::1 dev test-dummy proto kernel metric 0 pref
medium<br>
local fdde:33:44::1 dev test-dummy proto kernel metric 0 pref
medium<br>
<br>
# ip r sh t 10 | grep local<br>
local 10.20.30.1 dev test-dummy proto kernel scope host src
10.20.30.1<br>
local 10.20.40.1 dev test-dummy proto kernel scope host src
10.20.40.1<br>
<br>
But when I restart systemd-networkd (systemctl restart
systemd-networkd), the local route for the /128 IPv6 on the dummy
interface in the VRF table vanished:<br>
# ip r sh t 10 | grep local<br>
local 10.20.30.1 dev test-dummy proto kernel scope host src
10.20.30.1<br>
local 10.20.40.1 dev test-dummy proto kernel scope host src
10.20.40.1<br>
<br>
# ip -6 r sh t 10 | grep local<br>
local fdde:33:44::1 dev test-dummy proto kernel metric 0 pref
medium<br>
<br>
I am able to ping all addresses on the dummy interface except the
/128 IPv6 one:<br>
# ip vrf exec test-vrf ping fdde:11:22::1<br>
PING fdde:11:22::1(fdde:11:22::1) 56 data bytes<br>
^C<br>
--- fdde:11:22::1 ping statistics ---<br>
2 packets transmitted, 0 received, 100% packet loss, time
1000ms<br>
<br>
To fix this, I either have to delete the test-dummy interface and
restart systemd-networkd afterwards (ip l del test-dummy
&& systemctl restart systemd-networkd),<br>
or add this local route by hand again: ip -6 r add local
fdde:11:22::1 dev test-dummy proto kernel metric 0 pref medium
table 10<br>
<br>
The metric gets changed to 1024 though so I have to delete the
non-local route for this interface in order to be able to ping it
again so I prefer the first approach:<br>
# ip vrf exec test-vrf ping fdde:11:22::1<br>
PING fdde:11:22::1(fdde:11:22::1) 56 data bytes<br>
^C<br>
--- fdde:11:22::1 ping statistics ---<br>
1 packets transmitted, 0 received, 100% packet loss, time 0ms<br>
<br>
# ip -6 r sh t 10<br>
fdde:11:22::1 dev test-dummy proto kernel metric 256 pref
medium<br>
local fdde:11:22::1 dev test-dummy proto kernel metric 1024
pref medium<br>
<br>
# ip -6 r del fdde:11:22::1 dev test-dummy table 10<br>
<br>
# ip vrf exec test-vrf ping fdde:11:22::1<br>
PING fdde:11:22::1(fdde:11:22::1) 56 data bytes<br>
64 bytes from fdde:11:22::1: icmp_seq=1 ttl=64 time=0.038 ms<br>
<br>
I was able to reproduce this behavior with a Wireguard interface
aswell, so I think this behavior applies to all netdev types. Also
worth to mention, "networkctl reload" does not trigger this
behavior aswell.<br>
<br>
The kernel docs say:<br>
<br>
Local and connected routes for enslaved devices are
automatically moved to<br>
the table associated with VRF device. Any additional routes
depending on<br>
the enslaved device are dropped and will need to be reinserted
to the VRF<br>
FIB table following the enslavement.<br>
<br>
When commenting out the "VRF=test-vrf" on the dummy's .network
file and enslaving it by hand (ip l set dev test-dummy master
test-vrf), this works aswell as expected until I restart
systemd-networkd and I have to enslave it again and do above
steps.<br>
<br>
Am I missing something out or did I hit a bug here? Version is
systemd 245 (245.5-2-arch) on 5.6.13-arch1-1 (Arch Linux).<br>
<br>
Kind regards,<br>
<br>
Marcel Menzel<br>
</font>
</body>
</html>