[PATCH] drm/amdgpu: allow pinning DMA-bufs into VRAM if all importers can do P2P

[Christian König]

Am 28.03.25 um 20:14 schrieb Alex Deucher:
> On Thu, Jan 9, 2025 at 12:07 PM Felix Kuehling <felix.kuehling at amd.com> wrote:
>> From: Christian König <christian.koenig at amd.com>
>>
>> Try pinning into VRAM to allow P2P with RDMA NICs without ODP
>> support if all attachments can do P2P. If any attachment can't do
>> P2P just pin into GTT instead.
>>
>> Signed-off-by: Christian König <christian.koenig at amd.com>
>> Signed-off-by: Felix Kuehling <felix.kuehling at amd.com>
>> Reviewed-by: Felix Kuehling <felix.kuehling at amd.com>
>> Tested-by: Pak Nin Lui <pak.lui at amd.com>
>> Cc: Simona Vetter <simona.vetter at ffwll.ch>
> Applied along with the cgroups patch for amdgpu.

Nice, going to update the internal people as well.

Christian.

>
> Alex
>
>> ---
>>  drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 25 +++++++++++++++------
>>  1 file changed, 18 insertions(+), 7 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
>> index 8e81a83d37d84..83390143c2e9f 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
>> @@ -72,11 +72,25 @@ static int amdgpu_dma_buf_attach(struct dma_buf *dmabuf,
>>   */
>>  static int amdgpu_dma_buf_pin(struct dma_buf_attachment *attach)
>>  {
>> -       struct drm_gem_object *obj = attach->dmabuf->priv;
>> -       struct amdgpu_bo *bo = gem_to_amdgpu_bo(obj);
>> +       struct dma_buf *dmabuf = attach->dmabuf;
>> +       struct amdgpu_bo *bo = gem_to_amdgpu_bo(dmabuf->priv);
>> +       u32 domains = bo->preferred_domains;
>>
>> -       /* pin buffer into GTT */
>> -       return amdgpu_bo_pin(bo, AMDGPU_GEM_DOMAIN_GTT);
>> +       dma_resv_assert_held(dmabuf->resv);
>> +
>> +       /*
>> +        * Try pinning into VRAM to allow P2P with RDMA NICs without ODP
>> +        * support if all attachments can do P2P. If any attachment can't do
>> +        * P2P just pin into GTT instead.
>> +        */
>> +       list_for_each_entry(attach, &dmabuf->attachments, node)
>> +               if (!attach->peer2peer)
>> +                       domains &= ~AMDGPU_GEM_DOMAIN_VRAM;
>> +
>> +       if (domains & AMDGPU_GEM_DOMAIN_VRAM)
>> +               bo->flags |= AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
>> +
>> +       return amdgpu_bo_pin(bo, domains);
>>  }
>>
>>  /**
>> @@ -131,9 +145,6 @@ static struct sg_table *amdgpu_dma_buf_map(struct dma_buf_attachment *attach,
>>                 r = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);
>>                 if (r)
>>                         return ERR_PTR(r);
>> -
>> -       } else if (bo->tbo.resource->mem_type != TTM_PL_TT) {
>> -               return ERR_PTR(-EBUSY);
>>         }
>>
>>         switch (bo->tbo.resource->mem_type) {
>> --
>> 2.34.1
>>