[PATCH 2/2] drm/amdgpu: Synchronize after mapping into a compute VM

[Chen, Xiaogang]

On 2/24/2023 5:36 PM, Felix Kuehling wrote:
> Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
>
>
> Compute VMs use user mode queues for command submission. They cannot use
> a CS ioctl to synchronize with pending PTE updates and flush TLBs. Do
> this synchronization in amdgpu_gem_va_ioctl for compute VMs.
>
> Signed-off-by: Felix Kuehling <Felix.Kuehling at amd.com>
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 16 ++++++++++++++--
>   1 file changed, 14 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> index 6936cd63df42..7de5057c40ec 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> @@ -601,7 +601,7 @@ int amdgpu_gem_metadata_ioctl(struct drm_device *dev, void *data,
>   static void amdgpu_gem_va_update_vm(struct amdgpu_device *adev,
>                                      struct amdgpu_vm *vm,
>                                      struct amdgpu_bo_va *bo_va,
> -                                   uint32_t operation)
> +                                   uint32_t operation, uint32_t flags)
>   {
>          int r;
>
> @@ -620,6 +620,18 @@ static void amdgpu_gem_va_update_vm(struct amdgpu_device *adev,
>          }
>
>          r = amdgpu_vm_update_pdes(adev, vm, false);
> +       if (r)
> +               goto error;
> +
> +       if (vm->is_compute_context) {
> +               if (bo_va->last_pt_update)
> +                       r = dma_fence_wait(bo_va->last_pt_update, true);
> +               if (!r && vm->last_update)
> +                       r = dma_fence_wait(vm->last_update, true);
> +               if (!r)
> +                       r = amdgpu_amdkfd_flush_tlb(adev, vm,
> +                           TLB_FLUSH_LEGACY);

When kfd flush its vm tlb it uses TLB_FLUSH_LEGACY after map and 
TLB_FLUSH_HEAVYWEIGHT after unmap. Here use amdgpu_gen_va_ioctl to 
map/unmap buffer in kfd vm. Should we keep consistent with kfd:

use TLB_FLUSH_LEGACY if args->operation is AMDGPU_VA_OP_MAP, uses 
TLB_FLUSH_HEAVYWEIGHT if args->operation is AMDGPU_VA_OP_UNMAP?

> +       }
>
>   error:
>          if (r && r != -ERESTARTSYS)
> @@ -789,7 +801,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
>          }
>          if (!r && !(args->flags & AMDGPU_VM_DELAY_UPDATE) && !amdgpu_vm_debug)
>                  amdgpu_gem_va_update_vm(adev, &fpriv->vm, bo_va,
> -                                       args->operation);
> +                                       args->operation, args->flags);
>
>   error_backoff:
>          ttm_eu_backoff_reservation(&ticket, &list);
> --
> 2.34.1
>