[Intel-gfx] [PATCH v5 00/10] Enhance vfio PCI hot reset for vfio cdev device

[Xu, Terrence]

> -----Original Message-----
> From: Liu, Yi L <yi.l.liu at intel.com>
> Subject: [PATCH v5 00/10] Enhance vfio PCI hot reset for vfio cdev device
> 
> VFIO_DEVICE_PCI_HOT_RESET requires user to pass an array of group fds to
> prove that it owns all devices affected by resetting the calling device. While
> for cdev devices, user can use an iommufd-based ownership checking model
> and invoke VFIO_DEVICE_PCI_HOT_RESET with a zero-length fd array.
> 
> This series first creates iommufd_access for noiommu devices to fill the gap
> for adding iommufd-based ownership checking model, then extends
> VFIO_DEVICE_GET_PCI_HOT_RESET_INFO to check ownership and return
> the check result and the devid of affected devices to user. In the end,
> extends the VFIO_DEVICE_PCI_HOT_RESET to accept zero-length fd array for
> hot-reset with cdev devices.
> 
> The new hot reset method and updated _INFO ioctl are tested with the
> below qemu:
> 
> https://github.com/yiliu1765/qemu/tree/iommufd_rfcv4.mig.reset.v4_var3
> (requires to test with the cdev kernel)
> 
> Change log:
> 
> v5:
>  - Drop patch 01 of v4 (Alex)
>  - Create noiommu_access for noiommu devices (Jason)
>  - Reserve all negative iommufd IDs, hence VFIO can encode negative
>    values (Jason)
>  - Make vfio_iommufd_physical_devid() return -EINVAL if it's not called
>    with a physical device or a noiommu device.
>  - Add vfio_find_device_in_devset() in vfio_main.c (Alex)
>  - Add iommufd_ctx_has_group() to replace
> vfio_devset_iommufd_has_group().
>    Reason: vfio_devset_iommufd_has_group() only loops the devices within
>    the given devset to check the iommufd an iommu_group, but an
> iommu_group
>    can span into multiple devsets. So if failed to find the group in a
>    devset doesn't mean the group is not owned by the iommufd. So here
> either
>    needs to search all the devsets or add an iommufd API to check it. It
>    appears an iommufd API makes more sense.
>  - Adopt suggestions from Alex on patch 08 and 09 of v4, refine the hot-reset
>    uapi description and minor tweaks
>  - Use bitfields for bool members (Alex)
> 
> v4: https://lore.kernel.org/kvm/20230426145419.450922-1-yi.l.liu@intel.com/
>  - Rename the patch series subject
>  - Patch 01 is moved from the cdev series
>  - Patch 02, 06 are new per review comments in v3
>  - Patch 03/04/05/07/08/09 are from v3 with updates
> 
> v3: https://lore.kernel.org/kvm/20230401144429.88673-1-yi.l.liu@intel.com/
>  - Remove the new _INFO ioctl of v2, extend the existing _INFO ioctl to
>    report devid (Alex)
>  - Add r-b from Jason
>  - Add t-b from Terrence Xu and Yanting Jiang (mainly regression test)
> 
> v2: https://lore.kernel.org/kvm/20230327093458.44939-1-yi.l.liu@intel.com/
>  - Split the patch 03 of v1 to be 03, 04 and 05 of v2 (Jaon)
>  - Add r-b from Kevin and Jason
>  - Add patch 10 to introduce a new _INFO ioctl for the usage of device
>    fd passing usage in cdev path (Jason, Alex)
> 
> v1: https://lore.kernel.org/kvm/20230316124156.12064-1-yi.l.liu@intel.com/
> 
> Regards,
> 	Yi Liu
> 
> Yi Liu (10):
>   vfio-iommufd: Create iommufd_access for noiommu devices
>   vfio/pci: Update comment around group_fd get in
>     vfio_pci_ioctl_pci_hot_reset()
>   vfio/pci: Move the existing hot reset logic to be a helper
>   vfio: Mark cdev usage in vfio_device
>   iommufd: Reserve all negative IDs in the iommufd xarray
>   vfio-iommufd: Add helper to retrieve iommufd_ctx and devid for
>     vfio_device
>   vfio: Add helper to search vfio_device in a dev_set
>   iommufd: Add iommufd_ctx_has_group()
>   vfio/pci: Extend VFIO_DEVICE_GET_PCI_HOT_RESET_INFO for vfio device
>     cdev
>   vfio/pci: Allow passing zero-length fd array in
>     VFIO_DEVICE_PCI_HOT_RESET
> 
>  drivers/iommu/iommufd/device.c   |  53 +++++++++
>  drivers/iommu/iommufd/main.c     |   2 +-
>  drivers/vfio/iommufd.c           |  63 ++++++++++-
>  drivers/vfio/pci/vfio_pci_core.c | 184 ++++++++++++++++++++++++-------
>  drivers/vfio/vfio_main.c         |  15 +++
>  include/linux/iommufd.h          |  14 +++
>  include/linux/vfio.h             |  23 ++++
>  include/uapi/linux/vfio.h        |  60 +++++++++-
>  8 files changed, 368 insertions(+), 46 deletions(-)
> 
> --
> 2.34.1

The new uapi works fine.
Tested GVT-g / GVT-d VFIO legacy mode / compat mode / cdev mode, including negative tests. No regression be introduced.  

Tested-by: Terrence Xu <terrence.xu at intel.com>