[PATCH] drm/amdkfd: update SIMD distribution algo for GFXIP 9.4.2 onwards

[Greathouse, Joseph]

[AMD Official Use Only - General]

> -----Original Message-----
> From: Kuehling, Felix <Felix.Kuehling at amd.com>
> Sent: Friday, February 2, 2024 10:21 AM
> To: Bhardwaj, Rajneesh <Rajneesh.Bhardwaj at amd.com>; amd-gfx at lists.freedesktop.org
> Cc: Greathouse, Joseph <Joseph.Greathouse at amd.com>
> Subject: Re: [PATCH] drm/amdkfd: update SIMD distribution algo for GFXIP 9.4.2 onwards
>
>
> On 2024-02-01 13:54, Rajneesh Bhardwaj wrote:
> > In certain cooperative group dispatch scenarios the default SPI
> > resource allocation may cause reduced per-CU workgroup occupancy. Set
> > COMPUTE_RESOURCE_LIMITS.FORCE_SIMD_DIST=1 to mitigate soft hang
> > scenarions.
> >
> > Suggested-by: Joseph Greathouse <Joseph.Greathouse at amd.com>
> > Signed-off-by: Rajneesh Bhardwaj <rajneesh.bhardwaj at amd.com>
> > ---
> >   drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c | 4 ++++
> >   1 file changed, 4 insertions(+)
> >
> > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
> > b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
> > index 42d881809dc7..4b28e7dcb62f 100644
> > --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
> > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
> > @@ -303,6 +303,10 @@ static void update_mqd(struct mqd_manager *mm, void *mqd,
> >             update_cu_mask(mm, mqd, minfo, 0);
> >     set_priority(m, q);
> >
> > +   if (KFD_GC_VERSION(mm->dev) >= IP_VERSION(9, 4, 2))
>
> Are we sure this is only needed on GFX 9.4.2 and later GPUs? Does it affect older GFX 9.x GPUs as well? What about GFX 10 and 11? It
> seems the register bit exists for all those GPUs?

On gfx9 devices, it is only necessary for GFX 9.4.2 and beyond. This was a side effect of the move from 10 wave-slots per SIMD to 8 wave-slots per SIMD.

Checking with the hardware group (and running some basic tests against the problem we saw on gfx9 parts), this should not be necessary for gfx10 parts, either those with 20 wave-slots per SIMD or 16.

Thanks,
-Joe

>
> Regards,
>    Felix
>
>
> > +           m->compute_resource_limits = q->is_gws ?
> > +                   COMPUTE_RESOURCE_LIMITS__FORCE_SIMD_DIST_MASK : 0;
> > +
> >     q->is_active = QUEUE_IS_ACTIVE(*q);
> >   }
> >