[PATCH rdma-core v4 0/6] Add user space dma-buf support

[Xiong, Jianxin]

Does this series look fine now? I would like to try resolving any remaining issues so we
Can catch the window for kernel 5.12.

Edward, do you have more opinions on the pyverbs related changes?

Thanks,
Jianxin

> -----Original Message-----
> From: Xiong, Jianxin <jianxin.xiong at intel.com>
> Sent: Wednesday, December 02, 2020 12:57 PM
> To: linux-rdma at vger.kernel.org; dri-devel at lists.freedesktop.org
> Cc: Xiong, Jianxin <jianxin.xiong at intel.com>; Doug Ledford <dledford at redhat.com>; Jason Gunthorpe <jgg at ziepe.ca>; Leon Romanovsky
> <leon at kernel.org>; Sumit Semwal <sumit.semwal at linaro.org>; Christian Koenig <christian.koenig at amd.com>; Vetter, Daniel
> <daniel.vetter at intel.com>; Edward Srouji <edwards at nvidia.com>
> Subject: [PATCH rdma-core v4 0/6] Add user space dma-buf support
> 
> This is the fourth version of the patch series. Change log:
> 
> v4:
> * Rework the cmake funciton rdma_cython_module to support both single
>   source (.pyx) and multiple source (.pyx + [.c]*) scenarios instead
>   of using two separate functions
> * Rename 'dri_*' to 'drm_*' for the dmabuf allocation interface
> * Add option to dmabuf allocation routine to allow allocation from GTT
>   instead of VRAM
> * Add proper CPU access flags when allocating dmabufs
> * Remove 'radeon' driver support from the dmabuf allocation routines
> * Add comand line arguments to the tests for selecting GPU unit and
>   setting the option for allocating from GTT
> 
> v3: https://www.spinics.net/lists/linux-rdma/msg98059.html
> * Add parameter 'iova' to the new ibv_reg_dmabuf_mr() API
> * Change the way of allocating dma-buf object - use /dev/dri/renderD*
>   instead of /dev/dri/card* and use GEM object instead of dumb buffer
> * Add cmake function to allow building modules with mixed cython and C
>   source files
> * Add new tests that use dma-buf MRs for send/recv and rdma traffic
> * Skip dma-buf tests on unsupported systems
> * Remove some use of random values in the new tests
> * Add dealloc() and close() methods to the new classes
> * Replace string.format with f-string in python code
> * Fix some coding style issues: spacing, indentation, typo, comments
> 
> v2: https://www.spinics.net/lists/linux-rdma/msg97936.html
> * Put the kernel header updates into a separate commit
> * Add comments for the data structure used in python ioctl calls
> * Fix issues related to symbol versioning
> * Fix styling issues: extra spaces, unncecessary variable, typo
> * Fix an inproper error code usage
> * Put the new op into ibv_context_ops instead if verbs_context
> 
> v1: https://www.spinics.net/lists/linux-rdma/msg97865.html
> * Add user space API for registering dma-buf based memory regions
> * Update pyverbs with the new API
> * Add new tests
> 
> This is the user space counter-part of the kernel patch set to add dma-buf support to the RDMA subsystem.
> 
> This series consists of six patches. The first patch updates the kernel headers for dma-buf support. Patch 2 adds the new API function and
> updates the man pages. Patch 3 implements the new API in the mlx5 provider. Patch 4 adds new class definitions to pyverbs for the new API.
> Patch 5 adds a set of new tests for the new API. Patch 6 fixes bug in the utility code of the tests.
> 
> Pull request at github: https://github.com/linux-rdma/rdma-core/pull/895
> 
> Jianxin Xiong (6):
>   Update kernel headers
>   verbs: Support dma-buf based memory region
>   mlx5: Support dma-buf based memory region
>   pyverbs: Add dma-buf based MR support
>   tests: Add tests for dma-buf based memory regions
>   tests: Bug fix for get_access_flags()
> 
>  buildlib/pyverbs_functions.cmake         |  78 ++++++---
>  debian/libibverbs1.symbols               |   2 +
>  kernel-headers/rdma/ib_user_ioctl_cmds.h |  14 ++
>  kernel-headers/rdma/ib_user_verbs.h      |  14 --
>  libibverbs/CMakeLists.txt                |   2 +-
>  libibverbs/cmd_mr.c                      |  38 +++++
>  libibverbs/driver.h                      |   7 +
>  libibverbs/dummy_ops.c                   |  11 ++
>  libibverbs/libibverbs.map.in             |   6 +
>  libibverbs/man/ibv_reg_mr.3              |  27 ++-
>  libibverbs/verbs.c                       |  18 ++
>  libibverbs/verbs.h                       |  11 ++
>  providers/mlx5/mlx5.c                    |   2 +
>  providers/mlx5/mlx5.h                    |   3 +
>  providers/mlx5/verbs.c                   |  22 +++
>  pyverbs/CMakeLists.txt                   |  11 +-
>  pyverbs/dmabuf.pxd                       |  15 ++
>  pyverbs/dmabuf.pyx                       |  73 ++++++++
>  pyverbs/dmabuf_alloc.c                   | 278 +++++++++++++++++++++++++++++++
>  pyverbs/dmabuf_alloc.h                   |  19 +++
>  pyverbs/libibverbs.pxd                   |   2 +
>  pyverbs/mr.pxd                           |   6 +
>  pyverbs/mr.pyx                           | 105 +++++++++++-
>  tests/args_parser.py                     |   4 +
>  tests/test_mr.py                         | 264 ++++++++++++++++++++++++++++-
>  tests/utils.py                           |  30 +++-
>  26 files changed, 1012 insertions(+), 50 deletions(-)  create mode 100644 pyverbs/dmabuf.pxd  create mode 100644 pyverbs/dmabuf.pyx
> create mode 100644 pyverbs/dmabuf_alloc.c  create mode 100644 pyverbs/dmabuf_alloc.h
> 
> --
> 1.8.3.1