[systemd-devel] Fwd: [PATCH] Add support for detecting NIC partitions on Dell Servers

[Jordan Hargrave]

On Tue, Nov 10, 2015 at 9:37 AM, Tom Gundersen <teg at jklm.no> wrote:
> On Tue, Nov 10, 2015 at 2:29 PM, Jordan Hargrave <jharg93 at gmail.com> wrote:
>> On Tue, Nov 10, 2015 at 4:53 AM, Kay Sievers <kay at vrfy.org> wrote:
>>> On Tue, Nov 10, 2015 at 5:49 AM, Jordan Hargrave <jharg93 at gmail.com> wrote:
>>>> Cleaned up linux coding style
>>>>
>>>> This patch will integrate some of the features of biosdevname into systemd.
>>>> The code detects the port and index for detecting NIC partitions. This creates
>>>> a new environment variable, ID_NET_NAME_PARTITION of the format
>>>> <PORT>_<PARTITION>
>>>>
>>>> The patch will also decode SMBIOS slot number for NIC, and store in the variable
>>>> ID_NET_NAME_SMBIOS_SLOT.  Systemd does not have a method for naming
>>>> ports on a multi-port card plugged into a slot.
>>>
>>> Again, I don't think systemd should carry an SMBIOS parser.
>>>
>>> Sorry,
>>> Kay
>>
>> From a customer usability standpoint, having the slot numbers as part
>> of systemd would be a very useful feature.
>
> Sure, but I think Kay's point was that the needed info should be
> exposed from the kernel in a sysattr, not be parsed from udev. Any
> reason this cannot be done that way?
>

The pci kernel maintainer was reluctant to put it in the kernel and
said systemd should parse it.  So a bit of circular argument here.
I agree, it would be better for it to be a kernel sysfs attr.

>>  The current method only
>> works for single-port NICs in a slot.  Multi-port NICs, especially
>> ones with SR-IOV or multiple partitions get garbled names like
>
> Just to make sure we are on the same page, when you say "garbled" you
> mean that the naming scheme is not the one you want, but there are no
> bugs here, right?
>

Ok, garbled maybe wasn't a good description :) But from a customer
usability standpoint, enp68s0f0 isn't useful.

>> enp4s0
>> enp4s1
>> enp4s0d1
>> enp4s0f1
>> enp4s0f2
>> enp4s0f3
>> enp4s0f4
>> enp4s0f5
>> enp4s0f6
>> enp4s0f7
>> enp4s0f1d1
>> enp4s0f2d1
>> enp4s0f3d1
>> enp4s0f4d1
>> enp4s0f5d1
>> enp4s0f6d1
>> enp4s0f7d1
>> enp4s1d1
>> enp68s0f0
>> enp68s0f1
>> enp69s0f0
>> enp69s0f1
>>
>> That's another annoying thing with systemd names, the bus number is
>> *decimal*.  lspci is in hex, so the customer has to do a conversion to
>> figure out even what PCI device that is.
>
> I guess too late to change that now.
>
>> All enp4xxxx are a dual-port NIC in Slot 3 with 8 SR-IOV devices.
>
> Hm, there are 17 devices listed, shouldn't there be 16 based on your
> description?
>
>> All enp68xx and enp69xxx are a single quad-port NIC in slot 2.
>> Systemd breaks here if trying to name using slot numbers with the
>> existing method.  As there are 4 devices under the slot with same
>> device numbers, systemd would name them
>> ens2f0
>> ens2f1
>> ens2f0
>> ens2f1
>>
>> Which causes name collision.  I was able to verify this as either they
>> got named:
>> ens2f0
>> ens2f1
>> enp69s0f0
>> enp69s0f1
>>
>> or
>> enp68s0f0
>> enp68s0f1
>> ens2f0
>> ens2f1
>>
>> at startup.
>>
>> That's the best feature of biosdevname, being able to tell which slot
>> the NIC is located just from the name.  Systemd still has some
>> limitations and/or bugs in this regard.
>
> So how would your proposed naming scheme look in the examples you
> gave? Is the information needed to generate the name taken from the
> device in question or any of its parent devices (but never from its
> siblings or other devices) and hence independent of probe-ordering?
>
> Cheers,
>
> Tom

I'd love to have biosdevname-compatible name scheme, that would be the
best solution from Dell's standpoint.  Our customers are using
biosdevname, which is going away in newer releases of RedHat kernels.
So we need a suitable replacement.  No point in having two different
naming schemes, but the current systemd name style does have some
limitations.  The customer could continue to use biosdevname=0|1 on
the kernel command line to change which naming scheme was used.

Biosdevname-compatible names.
p2p1_2
p2p2_1
p5p1

etc.

For Dell network cards, the DCM/NPAR strings have the info necessary
to differentiate port numbers without relying on siblings.
There's also a new algorithm I've come up with to determine port
numbers on non-Dell cards that only relies on parent bridge info.

Most dual port cards follow this scheme
xx:xx.x PCI bridge (secondary=Y)
Y:00.0 = Port 1
Y:00.1 = Port 2

Intel Quad-port cards follow this scheme:
xx:xx.x PCI bridge (secondary=A) (PCIE Upstream Port)
A:02.0 PCI bridge (secondary=B)
A:04.0 PCI bridge (secondary=C)
B:00.0 = Port 1
B:00.1 = Port 2
C:00.0 = Port 3
C:00.1 = Port 4

So you can calulate the port number given the device B:D:F and its
parent (or parent's parent) secondary bus number.