[RFC 4/7] drm/amdgpu: Allocate doorbell slot for user queue

[Alex Deucher]

On Fri, Dec 23, 2022 at 2:37 PM Shashank Sharma <shashank.sharma at amd.com> wrote:
>
> This patch allocates a doorbell slot in the bar, for the usermode queue.
> We are using the unique queue-id to get this slot from MES.

We should manage the doorbell BAR just like VRAM.  I had a set of
patches to convert doorbell memory to GEM objects.  The user should be
able to allocate doorbell memory via the GEM IOCTL just like VRAM or
GTT.  Then when the user calls the USERQ IOCTL, we can just look up
the GEM object from the handle and then calculate the doorbell offset
based on the offset of the GEM object from the start of the BAR.

Alex

>
> Cc: Alex Deucher <alexander.deucher at amd.com>
> Cc: Christian Koenig <christian.koenig at amd.com>
> Signed-off-by: Shashank Sharma <shashank.sharma at amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c | 28 +++++++++++++++++++
>  1 file changed, 28 insertions(+)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c
> index a91cc304cb9e..b566ce4cb7f0 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c
> @@ -50,6 +50,25 @@ amdgpu_userqueue_remove_index(struct amdgpu_device *adev, struct amdgpu_usermode
>      ida_simple_remove(&uqg->ida, queue->queue_id);
>  }
>
> +static int
> +amdgpu_userqueue_get_doorbell(struct amdgpu_device *adev,
> +                    struct amdgpu_usermode_queue *queue)
> +{
> +    int r;
> +    unsigned int doorbell_index;
> +
> +    r = amdgpu_mes_alloc_process_doorbells(adev, &doorbell_index);
> +       if (r < 0) {
> +        DRM_ERROR("Failed to allocate doorbell for user queue\n");
> +        return r;
> +    }
> +
> +    /* We are using qnique queue-id to get doorbell here */
> +    queue->doorbell_index = amdgpu_mes_get_doorbell_dw_offset_in_bar(adev,
> +                           doorbell_index, queue->queue_id);
> +    return 0;
> +}
> +
>  static void
>  amdgpu_userqueue_setup_mqd(struct amdgpu_device *adev, struct amdgpu_usermode_queue *queue)
>  {
> @@ -257,12 +276,21 @@ int amdgpu_userqueue_create(struct amdgpu_device *adev, struct drm_file *filp,
>          goto free_queue;
>      }
>
> +    r = amdgpu_userqueue_get_doorbell(adev, queue);
> +    if (r) {
> +        DRM_ERROR("Failed to create doorbell for queue\n");
> +        goto free_mqd;
> +    }
> +
>      ctx->userq = queue;
>      args->out.q_id = queue->queue_id;
>      args->out.flags = 0;
>      mutex_unlock(&adev->userq.userq_mutex);
>      return 0;
>
> +free_mqd:
> +    amdgpu_userqueue_destroy_mqd(queue);
> +
>  free_queue:
>      amdgpu_userqueue_remove_index(adev, queue);
>      mutex_unlock(&adev->userq.userq_mutex);
> --
> 2.34.1
>