[PATCH 1/3] drm/amdgpu: Don't pin VRAM without DMABUF_MOVE_NOTIFY

[Christian König]

Am 16.04.25 um 06:45 schrieb Felix Kuehling:
> Pinning of VRAM is for peer devices that don't support dynamic attachment
> and move notifiers. But it requires that all such peer devices are able to
> access VRAM via PCIe P2P. Any device without P2P access requires migration
> to GTT, which fails if the memory is already pinned for another peer
> device.
>
> Sharing between GPUs should not require pinning in VRAM. However, if
> DMABUF_MOVE_NOTIFY is disabled in the kernel build, even DMABufs shared
> between GPUs must be pinned, which can lead to failures and functional
> regressions on systems where some peer GPUs are not P2P accessible.
>
> Disable VRAM pinning if move notifiers are disabled in the kernel build
> to fix regressions when sharing BOs between GPUs.
>
> Signed-off-by: Felix Kuehling <felix.kuehling at amd.com>

Reviewed-by: Christian König <christian.koenig at amd.com> for this one here.

> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 17 ++++++++++++-----
>  1 file changed, 12 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> index 667080cc9ae1c..9abe592968ab3 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> @@ -81,14 +81,21 @@ static int amdgpu_dma_buf_pin(struct dma_buf_attachment *attach)
>  
>  	dma_resv_assert_held(dmabuf->resv);
>  
> -	/*
> -	 * Try pinning into VRAM to allow P2P with RDMA NICs without ODP
> +	/* Try pinning into VRAM to allow P2P with RDMA NICs without ODP
>  	 * support if all attachments can do P2P. If any attachment can't do
>  	 * P2P just pin into GTT instead.
> +	 *
> +	 * To avoid with conflicting pinnings between GPUs and RDMA when move
> +	 * notifiers are disabled, only allow pinning in VRAM when move
> +	 * notiers are enabled.
>  	 */
> -	list_for_each_entry(attach, &dmabuf->attachments, node)
> -		if (!attach->peer2peer)
> -			domains &= ~AMDGPU_GEM_DOMAIN_VRAM;
> +	if (!IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY)) {
> +		domains &= ~AMDGPU_GEM_DOMAIN_VRAM;
> +	} else {
> +		list_for_each_entry(attach, &dmabuf->attachments, node)
> +			if (!attach->peer2peer)
> +				domains &= ~AMDGPU_GEM_DOMAIN_VRAM;
> +	}
>  
>  	if (domains & AMDGPU_GEM_DOMAIN_VRAM)
>  		bo->flags |= AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;