<div dir="ltr">Ah, no problem thanks for the info!</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 8, 2020 at 12:17 PM Paul Menzel <<a href="mailto:pmenzel%2Bsystemd-devel@molgen.mpg.de">pmenzel+systemd-devel@molgen.mpg.de</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Dear Luke,<br>
<br>
<br>
On 2020-07-08 13:10, Luke Alexander wrote:<br>
<br>
> I had a look through some of the issues against the systemd github repo but<br>
> could not find any matching - there were a couple which look promising but<br>
> don't completely resolve our issue.<br>
> <br>
> Our issue is that we have a k8s (1.18, kube-router CNI) cluster comprised<br>
> of a number of Ubuntu 16.04 nodes, we are in the process if upgrading the<br>
> nodes to Ubuntu 18.04 - however after upgrading the first node and adding<br>
> it back to the cluster we observed high packet loss from other cluster<br>
> nodes to the 18.04 node as well as ping error messages when running a<br>
> continuous ping from the 18.04 node to another node in the cluster, eg:<br>
> <br>
> 64 bytes from <a href="http://10.8.11.1" rel="noreferrer" target="_blank">10.8.11.1</a>: icmp_seq=91 ttl=64 time=0.088 ms<br>
> 64 bytes from <a href="http://10.8.11.1" rel="noreferrer" target="_blank">10.8.11.1</a>: icmp_seq=92 ttl=64 time=0.076 ms<br>
> ping: sendmsg: No buffer space available<br>
> ping: sendmsg: No buffer space available<br>
> ping: sendmsg: No buffer space available<br>
> <br>
> Another observation is that one of our daemonset pods (promtail) could not<br>
> start.<br>
> <br>
> systemctl --version<br>
> systemd 237<br>
> +PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP<br>
> +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN<br>
> -PCRE2 default-hierarchy=hybrid<br>
> <br>
> After many days of troubleshooting various possibilities, changing network<br>
> cable, NIC, a different server, various sysctl values, I eventually tried<br>
> switching from systemd-networkd to NetworkManager as the backend via<br>
> netplan config. After rebooting the server with NetworkManager in place and<br>
> re-adding the node back to the k8s cluster, the packet loss disappeared as<br>
> did the ping errors and the daemonset pod (promtail) also started normally.<br>
> <br>
> I would like to use Ubuntu's default of systemd-networkd to handle our<br>
> NIC/route config - but cannot until I've figured out what in<br>
> systemd-networkd is breaking.<br>
> <br>
> Any help is much appreciated on this matter.<br>
<br>
Unfortunately, upstream only supports the last two upstream versions of<br>
systemd. Please report the issue to the Ubuntu bug tracker Launchpad [1].<br>
It would probably help to verify, if the issue is still present in the newest<br>
version. So maybe try Ubuntu 20.04 first to see if it’s fixed there.<br>
Otherwise, building systemd yourself in Ubuntu should be doable too.<br>
<br>
I am sorry for the unsatisfying answer.<br>
<br>
<br>
Kind regards,<br>
<br>
Paul<br>
<br>
<br>
[1]: <a href="https://bugs.launchpad.net/ubuntu/+source/systemd" rel="noreferrer" target="_blank">https://bugs.launchpad.net/ubuntu/+source/systemd</a><br>
<br>
</blockquote></div>