[PATCH 15/26] drm/amdgpu: avoid KIQ clashing with compute or KFD queues

[Felix Kuehling]

See comment inline [FK].

On 17-04-06 02:21 AM, Andres Rodriguez wrote:
> Instead of picking an arbitrary queue for KIQ, search for one according
> to policy. The queue must be unused.
>
> Also report the KIQ as an unavailable resource to KFD.
>
> In testing I ran into KCQ initialization issues when using pipes 2/3 of
> MEC2 for the KIQ. Therefore the policy disallows grabbing one of these.
>
> Signed-off-by: Andres Rodriguez <andresx7 at gmail.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu.h        | 23 +++++++++++++---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c |  8 ++++++
>  drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c      | 43 ++++++++++++++++++++++++------
>  drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c      | 42 ++++++++++++++++++++++++-----
>  4 files changed, 98 insertions(+), 18 deletions(-)
[snip]
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
> @@ -104,20 +104,28 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev)
>  			.num_pipe_per_mec = adev->gfx.mec.num_pipe_per_mec,
>  			.num_queue_per_pipe = adev->gfx.mec.num_queue_per_pipe
>  		};
>  
>  		/* this is going to have a few of the MSBs set that we need to
>  		 * clear */
>  		bitmap_complement(gpu_resources.queue_bitmap,
>  				  adev->gfx.mec.queue_bitmap,
>  				  KGD_MAX_QUEUES);
>  
> +		/* remove the KIQ bit as well */
> +		if (adev->gfx.kiq.ring.ready)
> +			clear_bit(amdgpu_queue_to_bit(adev,
> +						      adev->gfx.kiq.ring.me + 1,

[FK] Shouldn't this be adev->gfx.kiq.ring.me - 1?

> +						      adev->gfx.kiq.ring.pipe,
> +						      adev->gfx.kiq.ring.queue),
> +				  gpu_resources.queue_bitmap);
> +
>  		/* According to linux/bitmap.h we shouldn't use bitmap_clear if
>  		 * nbits is not compile time constant */
>  		last_valid_bit = adev->gfx.mec.num_mec
>  				* adev->gfx.mec.num_pipe_per_mec
>  				* adev->gfx.mec.num_queue_per_pipe;
>  		for (i = last_valid_bit; i < KGD_MAX_QUEUES; ++i)
>  			clear_bit(i, gpu_resources.queue_bitmap);
>  
>  		amdgpu_doorbell_get_kfd_info(adev,
>  				&gpu_resources.doorbell_physical_address,
[snip]