[PATCH rdma-core v7 4/6] pyverbs: Add dma-buf based MR support

[Xiong, Jianxin]

> -----Original Message-----
> From: Jason Gunthorpe <jgg at ziepe.ca>
> Sent: Monday, February 01, 2021 7:29 AM
> To: Daniel Vetter <daniel at ffwll.ch>
> Cc: Leon Romanovsky <leon at kernel.org>; Gal Pressman <galpress at amazon.com>; Xiong, Jianxin <jianxin.xiong at intel.com>; Yishai Hadas
> <yishaih at nvidia.com>; linux-rdma <linux-rdma at vger.kernel.org>; Edward Srouji <edwards at nvidia.com>; dri-devel <dri-
> devel at lists.freedesktop.org>; Christian Koenig <christian.koenig at amd.com>; Doug Ledford <dledford at redhat.com>; Vetter, Daniel
> <daniel.vetter at intel.com>
> Subject: Re: [PATCH rdma-core v7 4/6] pyverbs: Add dma-buf based MR support
> 
> On Mon, Feb 01, 2021 at 03:10:00PM +0100, Daniel Vetter wrote:
> > On Mon, Feb 1, 2021 at 7:16 AM Leon Romanovsky <leon at kernel.org> wrote:
> > >
> > > On Sun, Jan 31, 2021 at 05:31:16PM +0200, Gal Pressman wrote:
> > > > On 25/01/2021 21:57, Jianxin Xiong wrote:
> > > > > Define a new sub-class of 'MR' that uses dma-buf object for the
> > > > > memory region. Define a new class 'DmaBuf' as a wrapper for
> > > > > dma-buf allocation mechanism implemented in C.
> > > > >
> > > > > Update the cmake function for cython modules to allow building
> > > > > modules with mixed cython and c source files.
> > > > >
> > > > > Signed-off-by: Jianxin Xiong <jianxin.xiong at intel.com>
> > > > > buildlib/pyverbs_functions.cmake |  78 +++++++----
> > > > >  pyverbs/CMakeLists.txt           |  11 +-
> > > > >  pyverbs/dmabuf.pxd               |  15 +++
> > > > >  pyverbs/dmabuf.pyx               |  73 ++++++++++
> > > > >  pyverbs/dmabuf_alloc.c           | 278 +++++++++++++++++++++++++++++++++++++++
> > > > >  pyverbs/dmabuf_alloc.h           |  19 +++
> > > > >  pyverbs/libibverbs.pxd           |   2 +
> > > > >  pyverbs/mr.pxd                   |   6 +
> > > > >  pyverbs/mr.pyx                   | 105 ++++++++++++++-
> > > > >  9 files changed, 557 insertions(+), 30 deletions(-)  create
> > > > > mode 100644 pyverbs/dmabuf.pxd  create mode 100644
> > > > > pyverbs/dmabuf.pyx  create mode 100644 pyverbs/dmabuf_alloc.c
> > > > > create mode 100644 pyverbs/dmabuf_alloc.h
> > >
> > > <...>
> > >
> > > > > index 0000000..05eae75
> > > > > +++ b/pyverbs/dmabuf_alloc.c
> > > > > @@ -0,0 +1,278 @@
> > > > > +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
> > > > > +/*
> > > > > + * Copyright 2020 Intel Corporation. All rights reserved. See
> > > > > +COPYING file  */
> > > > > +
> > > > > +#include <stdio.h>
> > > > > +#include <stdlib.h>
> > > > > +#include <stdint.h>
> > > > > +#include <unistd.h>
> > > > > +#include <string.h>
> > > > > +#include <errno.h>
> > > > > +#include <drm/drm.h>
> > > > > +#include <drm/i915_drm.h>
> > > > > +#include <drm/amdgpu_drm.h>
> > > > > +#include <drm/radeon_drm.h>
> > > >
> > > > I assume these should come from the kernel headers package, right?
> > >
> > > This is gross, all kernel headers should be placed in
> > > kernel-headers/* and "update" script needs to be extended to take drm/* files too :(.
> >
> > drm kernel headers are in the libdrm package. You need that anyway for
> > doing the ioctls (if you don't hand-roll the restarting yourself).
> >
> > Also our userspace has gone over to just outright copying the driver
> > headers. Not the generic headers, but for the rendering side of gpus,
> > which is the topic here, there's really not much generic stuff.
> >
> > > Jianxin, are you fixing it?
> >
> > So fix is either to depend upon libdrm for building, or have copies of
> > the headers included in the package for the i915/amdgpu/radeon headers
> > (drm/drm.h probably not so good idea).
> 
> We should have a cmake test to not build the drm parts if it can't be built, and pyverbs should skip the tests.
> 

Yes, I will add a test for that. Also, on SLES, the headers could be under /usr/include/libdrm instead of /usr/include/drm. The make test should check that and use proper path.