[PATCH] drm/amdgpu: allow pinning DMA-bufs into VRAM if all importers can do P2P

[James Zhu]

On 2025-01-09 11:57, Felix Kuehling wrote:
> From: Christian König <christian.koenig at amd.com>
>
> Try pinning into VRAM to allow P2P with RDMA NICs without ODP
> support if all attachments can do P2P. If any attachment can't do
> P2P just pin into GTT instead.
>
> Signed-off-by: Christian König <christian.koenig at amd.com>
> Signed-off-by: Felix Kuehling <felix.kuehling at amd.com>
> Reviewed-by: Felix Kuehling <felix.kuehling at amd.com>
> Tested-by: Pak Nin Lui <pak.lui at amd.com>
> Cc: Simona Vetter <simona.vetter at ffwll.ch>
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 25 +++++++++++++++------
>   1 file changed, 18 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> index 8e81a83d37d84..83390143c2e9f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> @@ -72,11 +72,25 @@ static int amdgpu_dma_buf_attach(struct dma_buf *dmabuf,
>    */
>   static int amdgpu_dma_buf_pin(struct dma_buf_attachment *attach)
>   {
> -	struct drm_gem_object *obj = attach->dmabuf->priv;
> -	struct amdgpu_bo *bo = gem_to_amdgpu_bo(obj);
> +	struct dma_buf *dmabuf = attach->dmabuf;
> +	struct amdgpu_bo *bo = gem_to_amdgpu_bo(dmabuf->priv);
> +	u32 domains = bo->preferred_domains;
>   
> -	/* pin buffer into GTT */
> -	return amdgpu_bo_pin(bo, AMDGPU_GEM_DOMAIN_GTT);
> +	dma_resv_assert_held(dmabuf->resv);
> +
> +	/*
> +	 * Try pinning into VRAM to allow P2P with RDMA NICs without ODP
> +	 * support if all attachments can do P2P. If any attachment can't do
> +	 * P2P just pin into GTT instead.
> +	 */
> +	list_for_each_entry(attach, &dmabuf->attachments, node)
> +		if (!attach->peer2peer)
> +			domains &= ~AMDGPU_GEM_DOMAIN_VRAM;
[JZ] I think we can stop check once one  entry has no p2p support.
> +
> +	if (domains & AMDGPU_GEM_DOMAIN_VRAM)
> +		bo->flags |= AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
> +
> +	return amdgpu_bo_pin(bo, domains);
>   }
>   
>   /**
> @@ -131,9 +145,6 @@ static struct sg_table *amdgpu_dma_buf_map(struct dma_buf_attachment *attach,
>   		r = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);
>   		if (r)
>   			return ERR_PTR(r);
> -
> -	} else if (bo->tbo.resource->mem_type != TTM_PL_TT) {
> -		return ERR_PTR(-EBUSY);
[JZ] Not sure if kfd_mem_dmamap_dmabuf needs update also.
>   	}
>   
>   	switch (bo->tbo.resource->mem_type) {