[Intel-gfx] [PATCH v2 10/10] vfio/pci: Add VFIO_DEVICE_GET_PCI_HOT_RESET_GROUP_INFO

[Liu, Yi L]

> From: Alex Williamson <alex.williamson at redhat.com>
> Sent: Tuesday, March 28, 2023 11:18 PM
> 
> On Tue, 28 Mar 2023 15:00:42 +0000
> "Liu, Yi L" <yi.l.liu at intel.com> wrote:
> 
> > > From: Alex Williamson <alex.williamson at redhat.com>
> > > Sent: Tuesday, March 28, 2023 10:46 PM
> > >
> > > On Tue, 28 Mar 2023 14:38:12 +0000
> > > "Liu, Yi L" <yi.l.liu at intel.com> wrote:
> > >
> > > > > From: Alex Williamson <alex.williamson at redhat.com>
> > > > > Sent: Tuesday, March 28, 2023 10:26 PM
> > > > >
> > > > > On Tue, 28 Mar 2023 06:19:06 +0000
> > > > > "Tian, Kevin" <kevin.tian at intel.com> wrote:
> > > > >
> > > > > > > From: Liu, Yi L <yi.l.liu at intel.com>
> > > > > > > Sent: Tuesday, March 28, 2023 11:32 AM
> > > > > > >
> > > > > > > > From: Alex Williamson <alex.williamson at redhat.com>
> > > > > > > > Sent: Tuesday, March 28, 2023 3:26 AM
> > > > > > > >
> > > > > > > > Additionally, VFIO_DEVICE_GET_PCI_HOT_RESET_INFO has a
> flags
> > > arg
> > > > > that
> > > > > > > > isn't used, why do we need a new ioctl vs defining
> > > > > > > > VFIO_PCI_HOT_RESET_FLAG_IOMMUFD_DEV_ID.
> > > > > > >
> > > > > > > Sure. I can follow this suggestion. BTW. I have a doubt here. This
> > > new
> > > > > flag
> > > > > > > is set by user. What if in the future kernel has new extensions and
> > > needs
> > > > > > > to report something new to the user and add new flags to tell
> user?
> > > Such
> > > > > > > flag is set by kernel. Then the flags field may have two kinds of
> flags
> > > > > (some
> > > > > > > set by user while some set by kernel). Will it mess up the flags
> space?
> > > > > > >
> > > > > >
> > > > > > flags in a GET_INFO ioctl is for output.
> > > > > >
> > > > > > if user needs to use flags as input to select different type of info
> then it
> > > > > should
> > > > > > be split into multiple GET_INFO cmds.
> > > > >
> > > > > I don't know that that's actually a rule, however we don't currently
> > > > > test flags is zero for input, so in this case I think we are stuck with
> > > > > it only being for output.
> > > > >
> > > > > Alternatively, should VFIO_DEVICE_GET_PCI_HOT_RESET_INFO
> > > > > automatically
> > > > > return the dev_id variant of the output and set a flag to indicate this
> > > > > is the case when called on a device fd opened as a cdev?  Thanks,
> > > >
> > > > Personally I prefer that user asks for dev_id info explicitly. The major
> > > reason
> > > > that we return dev_id is that the group/bdf info is not enough for the
> > > device
> > > > fd passing case. But if qemu opens device by itself, the group/bdf info
> is
> > > still
> > > > enough. So a device opened as a cdev doesn't mean it should return
> > > dev_id,
> > > > it depends on if user has the bdf knowledge.
> > >
> > > But if QEMU opens the cdev, vs getting it from the group, does it make
> > > any sense to return a set of group-ids + bdf in the host-reset info?
> > > I'm inclined to think the answer is no.
> > >
> > > Per my previous suggestion, I think we should always return the bdf. We
> > > can't know if the user is accessing through an fd they opened
> > > themselves or were passed,
> >
> > Oh, yes. I'm convinced by this reason since only cdev mode supports
> device fd
> > passing. So I'll reuse the existing _INFO and let kernel set a flag to mark
> the returned
> > info is dev_id+bdf.
> >
> > A check. If the device that the _INFIO is invoked is opened via cdev, but
> there
> > are devices in the dev_set that are got via VFIO_GROUP_GET_DEVICE_FD,
> should
> > I fail it or allow it?
> 
> It's a niche case, but I think it needs to be allowed. 

I'm also wondering if it is common in the future. Actually, a user should be
preferred to either use the group or cdev, but not both. Otherwise, it looks
to be bad programming.:-)

Also, as an earlier remark from Jason. If there are affected devices that are
opened by other users, the new _INFO should fail with -EPERM. I know this
remark was for the new _INFO ioctl. But now, we are going to reuse the
existing _INFO, so I'd also want to check if we still need this policy? If yes,
then it is a problem to check the owner of the devices that are opened by
the group path.

https://lore.kernel.org/kvm/ZBsF950laMs2ldFc@nvidia.com/


> We'd still
> report the bdf for those devices, but make use of the invalid/null
> dev-id.  I think this empowers userspace that they could make the same
> call on a group opened fd if necessary.

For the devices opened via group path, it should have an entry that
includes invalid_dev_id+bdf. So user can map it to the device. But
there is no group_id, this may be fine since group is just a shortcut
to find the device. Is it?


> An alternative would be to
> redefine the returned data structure entirely with a flag per entry
> describing the output, but then I think we need to invent a kernel
> policy of which gets reported, which seems overly complicated if our
> goal is to phase out group usage.  Make sense, or will this bite us?

This does smell complicated. 😊 maybe minimum change is better as
the group is going to be phased out.

Regards,
Yi Liu