[PATCH] drm/amdkfd: Don't trigger evictions unmapping dmabuf attachments

[Eric Huang]

Reviewed-by: Eric Huang <jinhuieric.huang at amd.com>

Regards,
Eric

On 2023-05-01 16:52, Felix Kuehling wrote:
> Don't move DMABuf attachments for PCIe P2P mappings to the SYSTEM domain
> when unmapping. This avoids triggering eviction fences unnecessarily.
> Instead do the move to SYSTEM and back to GTT when mapping these
> attachments to ensure the SG table gets updated after evictions.
>
> This may still trigger unnecessary evictions if user mode unmaps and
> remaps the same BO. However, this is unlikely in real applications.
>
> Cc: Eric Huang <jinhuieric.huang at amd.com>
> Signed-off-by: Felix Kuehling <Felix.Kuehling at amd.com>
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 15 ++++++++++-----
>   1 file changed, 10 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> index 1002c7834386..bb8e6f6793c0 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> @@ -530,6 +530,12 @@ kfd_mem_dmamap_dmabuf(struct kfd_mem_attachment *attachment)
>   {
>   	struct ttm_operation_ctx ctx = {.interruptible = true};
>   	struct amdgpu_bo *bo = attachment->bo_va->base.bo;
> +	int ret;
> +
> +	amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_CPU);
> +	ret = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);
> +	if (ret)
> +		return ret;
>   
>   	amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_GTT);
>   	return ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);
> @@ -662,11 +668,10 @@ kfd_mem_dmaunmap_userptr(struct kgd_mem *mem,
>   static void
>   kfd_mem_dmaunmap_dmabuf(struct kfd_mem_attachment *attachment)
>   {
> -	struct ttm_operation_ctx ctx = {.interruptible = true};
> -	struct amdgpu_bo *bo = attachment->bo_va->base.bo;
> -
> -	amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_CPU);
> -	ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);
> +	/* This is a no-op. We don't want to trigger eviction fences when
> +	 * unmapping DMABufs. Therefore the invalidation (moving to system
> +	 * domain) is done in kfd_mem_dmamap_dmabuf.
> +	 */
>   }
>   
>   /**