[systemd-devel] systemd-resolved/NetworkManager resolv.conf handling

[Petr Menšík]

On 10/26/22 20:44, Thomas HUMMEL wrote:
> Hello,
>
> I'm not sure if this is a systemd-resolved or NetworkManager question 
> but it involves both (I know Thomas HALLER is a member of this list too)
>
> on
>
> Fedora release 36 (Thirty Six) using the following kernel and packages
>
>     5.19.16-200.fc36.x86_64 #1 SMP PREEMPT_DYNAMIC
>
>     systemd-250.8-1.fc36.x86_64
>     systemd-resolved-250.8-1.fc36.x86_64
>     NetworkManager-1.38.4-1.fc36.x86_64
>
> I'm using a proprietary vpn client which does not seem to work very 
> well with systemd-resolved. As a matter of fact it seems to create a 
> manual NM profile which does not include dns properties and it seems 
> to (try to) set /etc/resolv.conf aside (F5 vpn linux client f5fpc for 
> the record)
>
> Making it work is not the question here. I'm trying to understand how 
> the 2 nameservers it configures may end up in 
> /run/systemd/resolve/resolv.conf (and global systemd-resolved config 
> as shown by resolvectl status) ONLY when I switch from a non 
> systemd-resolved config then back to a systemd-resolved config

/etc/resolv.conf is usually symlink to either 
/run/systemd/resolve/resolv.conf or 
/run/systemd/resolve/stub-resolv.conf. These nameservers ends there, 
because the f5fpc client just rewritten /etc/resolv.conf with a content 
it thought is appropriate.

I think you should raise and issue to f5 support and request correct 
integration with at least Network Manager. If it had been told the dns 
servers it should use, it could propagate them to systemd-resolved. If 
it has already NM profile, I don't see a reason why DNS servers are not 
configured by it. It should allow at least by some configuration change 
to propagate those servers to NM. It should not overwrite 
/etc/resolv.conf, especially if it is just symlink to other place.

I would suggest using strace to find what exactly it does and what it 
tries to modify. I expect sources for that client are not available.

>
> Here's exactly what I'm doing/experiencing:
>
> Starting from
>
> a) default NetworkManager config:
>
> # grep -iE 'dns|rc\.manager' NetworkManager.conf
> # ls -l conf.d/
> total 0
>
> b) systemd-resolved stub-resolv.conf mode:
>
> # ls -l /etc/resolv.conf
> lrwxrwxrwx 1 root root 37 Oct 26 19:15 /etc/resolv.conf -> 
> /run/systemd/resolve/stub-resolv.conf
>
> and with (not linked from /etc/resolv.conf) :
>
> /run/systemd/resolve/resolve.conf following content:
>
> nameserver 192.168.1.1
> nameserver 2a01:cb00:7e1:3300:aa6a:bbff:fe6e:190
> search home
>
> matching my auto wireless NM profile
>
> 1) I start the vpn client
>
> obviously it does not work very well with systemd-resolved as I don't 
> get corresponding nameserver (10.33.1.2,10.33.1.3) anywhere and name 
> resolution does not work for corresponding zones
>
> /run/systemd/resolve/resolve.conf content has not changed
>
> 2) I stop the vpn client, and switch to the following setup
>
> # rm /etc/resolv.conf
> rm: remove symbolic link '/etc/resolv.conf'? y
>
> # cat <<EOF > /etc/NetworkManager/conf.d/foo.conf
> > [main]
> > dns=default
> > rc.manager=file
> > EOF
>
> # reboot
>
> -> after the reboot the /etc/resolv.conf link as been recreated : why ?
>
> (/run/systemd/resolve/resolv.conf hasn't changed, which seems normal 
> to me)
>
> 3) I remove it again and reboot
>
> # rm /etc/resolv.conf
> rm: remove symbolic link '/etc/resolv.conf'? y
>
> # reboot
The systemd guys believe the systemd-resolved should always create 
/etc/resolv.conf if it does not exist already. Create empty 
/etc/resolv.conf file as a workaround.
>
> -> this time /etc/resolv.conf is as expected a regular file which 
> content is handled by NM:
>
> $ ls -l /etc/resolv.conf
> -rw-r--r-- 1 root root 114 Oct 26 20:22 /etc/resolv.conf
> $ cat /etc/resolv.conf
> # Generated by NetworkManager
> search home
> nameserver 192.168.1.1
> nameserver 2a01:cb00:7e1:3300:aa6a:bbff:fe6e:190
>
>
> 4) I start the vpn client
>
> it wrote to /etc/resolv.conf (which seems wrong to me but is out of 
> scope here)
>
> $ cat /etc/resolv.conf
> #F5 Networks Inc. :File modified by VPN process
> search pasteur.fr home
> nameserver 10.33.1.2
> nameserver 10.33.1.3
>
> the 2 nameservers it provided do not appear in 
> /run/systemd/resolve/resolv.conf
>
> 6) I stop the vpn client switch back to my orgininal config, and reboot
>
> # rm /etc/NetworkManager/conf.d/foo.conf
> rm: remove regular file '/etc/NetworkManager/conf.d/foo.conf'? y
>
> # rm /etc/resolv.conf
> rm: remove regular file '/etc/resolv.conf'? y
>
> # ln -s /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf
>
> # reboot
>
> -> everything looks as expected
>
> 7) I start the vpn client
>
> -> its provided nameserver appear in /run/systemd/resolv/resolv.conf 
> (and resolution of related zones work)
>
> -> why ? Where does the info come from ?
>
> nameserver 10.33.1.2
> nameserver 10.33.1.3
> nameserver 192.168.1.1
> # Too many DNS servers configured, the following entries may be ignored.
> nameserver 2a01:cb00:7e1:3300:aa6a:bbff:fe6e:190
> search pasteur.fr home
>
> Can you help me figure out what's happening or at least how can the 
> behavior seem to change with what seem a rollback to the initial state ?

I just guess systemd-resolved might have detected outside change of 
resolv.conf and adds the values provided by F5 client to its servers 
set. I think systemd-resolved detects the file were modified by another 
process and rewrites it again. But first obtains nameservers in that 
changed file. Does it change resolvectl status output?

In any case please contact F5 client support and ask for at least 
working NM integration, including DNS servers provisioning. It would 
have the same problem with dns=dnsmasq plugin in NM, so it is not just 
systemd-resolved specific.

Does it show DNS servers on this command: nmcli connection show 
<F5connection> | grep .DNS

When the F5 client is connected?

>
> Thanks for your help
>
> -- 
> Thomas HUMMEL
>
-- 
Petr Menšík
Software Engineer, RHEL
Red Hat, https://www.redhat.com/
PGP: DFCF908DB7C87E8E529925BC4931CA5B6C9FC5CB