[systemd-devel] How to set up virtual network interface cards (NIC) with systemd-networkd "The Right Way (tm)"?

[M. Buecher]

Dear all,

I read multiple articles on the internet about virtual interfaces via 
systemd-networkd, but most article just list the config files and do not 
explain why they did something this or that way.
Most are using MACVLAN netdevs but I couldn't get them working 
correctly, although the ip addresses were available on the interfaces.
In the docs, FAQ and mailing list I couldn't find anything related to 
it.
I'm ok with most network topics, but unfortunately I'm not a network 
admin/expert, so please bear with me.


* Goal
A new mini pc shall become the gateway between all internal IP networks, 
DHCP server for the main internal IP network and the internal DNS server 
plus provide some additional DNS server instances for special cases.
For the DNS server scenario multiple additional virtual network 
interfaces are needed on the real network interface card (NIC) with 
systemd-networkd.
IP addresses on the real and virtual interfaces shall be reachable from 
other machines and from all real/virtual interfaces on the mini pc 
itself.
Linux System is Debian GNU/Linux 9.9 (stretch) with kernel 
4.9.0-3/4.9.30-2+deb9u5 and systemd 232 +PAM +AUDIT +SELINUX +IMA 
+APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ 
+LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN
The old "networking.service" (/etc/network/interface*) is disabled and 
"systemd-networkd.service" enabled.

In the first step the solution shall be implemented in a pure IPv4 
scenario with no firewall on the mini pc itself, later in a dual stack 
scenario plus ip[6]tables firewall.

The real NIC is named "ens192" and the virtual interfaces are named 
"dnsextra01" and "dnsextra02".
IPv4 LAN #1 is 192.168.1.0/24 with default gateway 192.168.1.254 (via 
router device) to internet plus gateway 192.168.1.50 (mini pc) to IPv4 
LAN #2.
IPv4 LAN #2 is 192.168.2.0/24 with gateway plus dns 192.168.2.1 (mini 
pc).


* Detailed feature list
a) "ens192" has the main IPv4 LAN #1 with 192.168.1.50/24 and secondary 
IPv4 LAN #2 with 192.168.2.1/24.
For IPv4 LAN #2 it is also the gateway to IPv4 LAN #1 and the internet.
It provides the main DNS server instance for both IPv4 LANs.

b) "dnsextra01" (.98) has the main IPv4 LAN #1 with 192.168.1.98/24 and 
secondary IPv4 LAN #2 with 192.168.2.98/24.
It provides a special case DNS server instance for some machines in both 
IPv4 LANs.

c) "dnsextra02" (.99) has only the main IPv4 LAN #1 with 192.168.1.99/24
It provides a special case DNS server instance for one machine in IPv4 
LAN #1.

d) All machines in both IPv4 LANs should be able to ping all IP 
addresses of all real/virtual interfaces.
ping -O -c 10 
<192.168.1.50|192.168.1.98|192.168.1.99|192.168.2.1|192.168.2.98>

e) All real/virtual interfaces should be able to ping all IP addresses 
of all other real/virtual interfaces.
ping -I ens192 -O -c 10 <192.168.1.98|192.168.1.99|192.168.2.98>
ping -I dnsextra01 -O -c 10 <192.168.1.50|192.168.1.99|192.168.2.1>
ping -I dnsextra01 -O -c 10 
<192.168.1.50|192.168.1.98|192.168.2.1|192.168.2.98>


* My try
The following setup allows to ping some IPv4 addresses from other 
machines, but only sometimes and then it also takes several seconds 
until a ping finally succeeds.
Pinging the other interfaces on the mini pc itself does NOT work at all.
If the netdevs via MACVLAN are disabled, then the mini pc reacts nearly 
instantly on network requests (e.g. ssh, ping) and forwarding from IPv4 
LAN #1 to LAN #2 works fine.

a) /etc/sysctl.d/90_ipv4_filter.conf
net.ipv4.conf.all.arp_filter=1
net.ipv4.conf.all.rp_filter=1

b) /etc/systemd/network/ens192.network
[Match]
Name=ens192

[Network]
IPForward=yes
LinkLocalAddressing=ipv6
IPv6AcceptRA=yes
IPv6PrivacyExtensions=yes

## Virtual NICs on ens192
MACVLAN=dnsextra01
MACVLAN=dnsextra02

Address=192.168.1.50/24
Address=192.168.2.1/24

Gateway=192.168.1.254

c) /etc/systemd/network/dnsextra01.netdev
[NetDev]
Name=dnsextra01
Kind=macvlan

[MACVLAN]
Mode=bridge

d) /etc/systemd/network/dnsextra01.network
[Match]
Name=dnsextra01

[Network]
IPForward=yes
Address=192.168.1.98/24
Address=192.168.2.98/24

e) dnsextra02 same as dnsextra01 just only 192.168.2.99/24


What is wrong in this setup? How should this be done correctly via 
systemd-networkd?
Is a newer version of systemd needed for this to work?

Any help is greatly appreciated.
Matthias "Maddes" Bücher