Gang submit

[Christian König]

Hi Monk,

> If one gfx ib and one compute ib are in a gang, can they run literally  in parallel on GPU ?

Yes, that is essentially the functionality of gang submit.

The driver stack must guarantee that those IBs run at the same time 
because they use a ring buffer to communicate with each other.

> if one gfx ib and one compute ib are belong to two gang, they will be put to the gfx and compute ring one by one (e.g:  gang1-gfx-ib scheduled and signaled, and then gang2-compute-ib scheduled )

Yes, gang submission should never overlap or otherwise you can run into 
lockups.

Regards,
Christian.

Am 06.09.22 um 03:43 schrieb Liu, Monk:
> [AMD Official Use Only - General]
>
> Hi Christian
>
>
>> A gang submission guarantees that multiple IBs can run on different engines at the same time.
>> The effect is that as long as members of a gang are waiting to be submitted no other gang can start pushing jobs to the hardware and so deadlocks are effectively prevented.
> Could you please help to explain or confirm:
>
> 1) If one gfx ib and one compute ib are in a gang, can they run literally  in parallel on GPU ?
> 2) if one gfx ib and one compute ib are belong to two gang, they will be put to the gfx and compute ring one by one (e.g:  gang1-gfx-ib scheduled and signaled, and then gang2-compute-ib scheduled )
>
> Thanks
> -------------------------------------------------------------------
> Monk Liu | Cloud GPU & Virtualization Solution | AMD
> -------------------------------------------------------------------
> we are hiring software manager for CVS core team
> -------------------------------------------------------------------
>
> -----Original Message-----
> From: amd-gfx <amd-gfx-bounces at lists.freedesktop.org> On Behalf Of Christian König
> Sent: 2022年3月3日 16:23
> To: amd-gfx at lists.freedesktop.org; Olsak, Marek <Marek.Olsak at amd.com>
> Subject: Gang submit
>
> Hi guys,
>
> this patch set implements the the requirement for so called gang submissions in the CS interface.
>
> A gang submission guarantees that multiple IBs can run on different engines at the same time.
>
> This is implemented by keeping a global per-device gang around represented by a dma_fence which signals as soon as all jobs in a gang are pushed to the hardware.
>
> The effect is that as long as members of a gang are waiting to be submitted no other gang can start pushing jobs to the hardware and so deadlocks are effectively prevented.
>
> The whole set is based on top of my dma_resv_usage work and a few patches merged over from amd-staging-drm-next, so it won't easily apply anywhere.
>
> Please review and comment,
> Christian.
>