[PATCH v9 00/14] AMDGPU usermode queues

[Alex Deucher]

On Fri, Apr 26, 2024 at 10:17 AM Shashank Sharma
<shashank.sharma at amd.com> wrote:
>
> This patch series introduces AMDGPU usermode queues for gfx workloads.
> Usermode queues is a method of GPU workload submission into the graphics
> hardware without any interaction with kernel/DRM schedulers. In this
> method, a userspace graphics application can create its own workqueue and
> submit it directly in the GPU HW.
>
> The general idea of how this is supposed to work:
> - The application creates the following GPU objetcs:
>   - A queue object to hold the workload packets.
>   - A read pointer object.
>   - A write pointer object.
>   - A doorbell page.
>   - Shadow bufffer pages.
>   - GDS buffer pages (as required).
> - The application picks a 32-bit offset in the doorbell page for this
>   queue.
> - The application uses the usermode_queue_create IOCTL introduced in
>   this patch, by passing the GPU addresses of these objects (read ptr,
>   write ptr, queue base address, shadow, gds) with doorbell object and
>   32-bit doorbell offset in the doorbell page.
> - The kernel creates the queue and maps it in the HW.
> - The application maps the GPU buffers in process address space.
> - The application can start submitting the data in the queue as soon as
>   the kernel IOCTL returns.
> - After filling the workload data in the queue, the app must write the
>   number of dwords added in the queue into the doorbell offset and the
>   WPTR buffer, and the GPU will start fetching the data.
> - This series adds usermode queue support for all three MES based IPs
>   (GFX, SDMA and Compute).

I think we also need a new INFO IOCTL query to get the doorbell
offsets for each engine type within each doorbell page.

Alex

>
> libDRM changes for this series and a sample DRM test program can be found
> in the MESA merge request here:
> https://gitlab.freedesktop.org/mesa/drm/-/merge_requests/287
>
> Alex Deucher (1):
>   drm/amdgpu: UAPI for user queue management
>
> Arvind Yadav (1):
>   drm/amdgpu: enable compute/gfx usermode queue
>
> Shashank Sharma (12):
>   drm/amdgpu: add usermode queue base code
>   drm/amdgpu: add new IOCTL for usermode queue
>   drm/amdgpu: add helpers to create userqueue object
>   drm/amdgpu: create MES-V11 usermode queue for GFX
>   drm/amdgpu: create context space for usermode queue
>   drm/amdgpu: map usermode queue into MES
>   drm/amdgpu: map wptr BO into GART
>   drm/amdgpu: generate doorbell index for userqueue
>   drm/amdgpu: cleanup leftover queues
>   drm/amdgpu: fix MES GFX mask
>   drm/amdgpu: enable SDMA usermode queues
>   drm/amdgpu: add kernel config for gfx-userqueue
>
>  drivers/gpu/drm/amd/amdgpu/Kconfig            |   8 +
>  drivers/gpu/drm/amd/amdgpu/Makefile           |   7 +
>  drivers/gpu/drm/amd/amdgpu/amdgpu.h           |   3 +
>  drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c       |   2 +
>  drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c       |   6 +
>  drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c       |   3 -
>  drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h       |   1 -
>  drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c | 296 ++++++++++++++++
>  drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c        |  10 +
>  drivers/gpu/drm/amd/amdgpu/mes_v10_1.c        |   9 +-
>  drivers/gpu/drm/amd/amdgpu/mes_v11_0.c        |   9 +-
>  .../gpu/drm/amd/amdgpu/mes_v11_0_userqueue.c  | 317 ++++++++++++++++++
>  drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c        |   6 +
>  .../gpu/drm/amd/include/amdgpu_userqueue.h    |  79 +++++
>  include/uapi/drm/amdgpu_drm.h                 | 111 ++++++
>  15 files changed, 859 insertions(+), 8 deletions(-)
>  create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c
>  create mode 100644 drivers/gpu/drm/amd/amdgpu/mes_v11_0_userqueue.c
>  create mode 100644 drivers/gpu/drm/amd/include/amdgpu_userqueue.h
>
> --
> 2.43.2
>