[PATCH 5/8] drm/amdgpu/mes12: add support for setting gang submit

[Alex Deucher]

On Tue, Apr 29, 2025 at 2:56 PM Liu, Shaoyun <Shaoyun.Liu at amd.com> wrote:
>
> [AMD Official Use Only - AMD Internal Distribution Only]
>
> Correctly , the gang submission related stuff only used for scheduler pipe , kiq is for used for kernel driver only and  should not expose to the  user queue.

What about kernel queues mapped through MES?  Do those need to go
through KIQ or is SCHED ok?  They seem to go through KIQ today.  Is
going through SCHED ok?

Alex


>
> Regards
> Shaoyun.liu
>
> -----Original Message-----
> From: amd-gfx <amd-gfx-bounces at lists.freedesktop.org> On Behalf Of Alex Deucher
> Sent: Tuesday, April 29, 2025 2:39 PM
> To: Chen, Michael <Michael.Chen at amd.com>
> Cc: Deucher, Alexander <Alexander.Deucher at amd.com>; amd-gfx at lists.freedesktop.org; Khatri, Sunil <Sunil.Khatri at amd.com>; Liang, Prike <Prike.Liang at amd.com>
> Subject: Re: [PATCH 5/8] drm/amdgpu/mes12: add support for setting gang submit
>
> On Tue, Apr 29, 2025 at 2:23 PM Chen, Michael <Michael.Chen at amd.com> wrote:
> >
> > [Public]
> >
> >
> >
> >
> > ________________________________
> > From: amd-gfx <amd-gfx-bounces at lists.freedesktop.org> on behalf of
> > Alex Deucher <alexander.deucher at amd.com>
> > Sent: Monday, April 28, 2025 5:20 PM
> > To: amd-gfx at lists.freedesktop.org <amd-gfx at lists.freedesktop.org>
> > Cc: Deucher, Alexander <Alexander.Deucher at amd.com>; Khatri, Sunil
> > <Sunil.Khatri at amd.com>; Liang, Prike <Prike.Liang at amd.com>
> > Subject: [PATCH 5/8] drm/amdgpu/mes12: add support for setting gang
> > submit
> >
> > Enable a primary and secondary queue that schedule together.
> >
> > v2: fix offset of api_status (Prike)
> >
> > Acked-by: Sunil Khatri <sunil.khatri at amd.com>
> > Reviewed-by: Prike Liang <Prike.Liang at amd.com>
> > Signed-off-by: Alex Deucher <alexander.deucher at amd.com>
> > ---
> >  drivers/gpu/drm/amd/amdgpu/mes_v12_0.c | 28
> > ++++++++++++++++++++++++++
> >  1 file changed, 28 insertions(+)
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
> > b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
> > index f9f2fbc0a7166..57d8b78210f9f 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
> > @@ -879,6 +879,33 @@ static int mes_v12_0_reset_hw_queue(struct amdgpu_mes *mes,
> >                          offsetof(union MESAPI__RESET, api_status));
> > }
> >
> > +static int mes_v12_0_set_gang_submit(struct amdgpu_mes *mes,
> > +                                    struct mes_set_gang_submit_input
> > +*input) {
> > +       union MESAPI__SET_GANG_SUBMIT mes_gang_submit_pkt;
> > +       int pipe;
> > +
> > +       memset(&mes_gang_submit_pkt, 0, sizeof(mes_gang_submit_pkt));
> > +
> > +       mes_gang_submit_pkt.header.type = MES_API_TYPE_SCHEDULER;
> > +       mes_gang_submit_pkt.header.opcode = MES_SCH_API_SET_GANG_SUBMIT;
> > +       mes_gang_submit_pkt.header.dwsize = API_FRAME_SIZE_IN_DWORDS;
> > +
> > +       mes_gang_submit_pkt.set_gang_submit.gang_context_addr =
> > +               input->primary_gang_context_addr;
> > +       mes_gang_submit_pkt.set_gang_submit.slave_gang_context_addr =
> > +               input->secondary_gang_context_addr;
> > +
> > +       if (mes->adev->enable_uni_mes)
> > +               pipe = AMDGPU_MES_KIQ_PIPE;
> > +       else
> > +               pipe = AMDGPU_MES_SCHED_PIPE;
> >
> > I think this packet should be always submitted to
> > AMDGPU_MES_SCHED_PIPE, same pipe as in mes_v12_0_add_hw_queue where the queue is created.
> > This is because HWS on AMDGPU_MES_KIQ_PIPE does not have the queue information.
>
> Sounds like the reset queue function and the map and unmap legacy queues should also be changed?
>
> Alex
>
> >
> > +
> > +       return mes_v12_0_submit_pkt_and_poll_completion(mes, pipe,
> > +                       &mes_gang_submit_pkt, sizeof(mes_gang_submit_pkt),
> > +                       offsetof(union MESAPI__SET_GANG_SUBMIT,
> > +api_status)); }
> > +
> >  static const struct amdgpu_mes_funcs mes_v12_0_funcs = {
> >          .add_hw_queue = mes_v12_0_add_hw_queue,
> >          .remove_hw_queue = mes_v12_0_remove_hw_queue, @@ -888,6
> > +915,7 @@ static const struct amdgpu_mes_funcs mes_v12_0_funcs = {
> >          .resume_gang = mes_v12_0_resume_gang,
> >          .misc_op = mes_v12_0_misc_op,
> >          .reset_hw_queue = mes_v12_0_reset_hw_queue,
> > +       .set_gang_submit = mes_v12_0_set_gang_submit,
> >  };
> >
> >  static int mes_v12_0_allocate_ucode_buffer(struct amdgpu_device
> > *adev,
> > --
> > 2.49.0
> >