[PATCH 4/7] drm/msm/A6xx: Implement preemption for A7XX targets

[Antonino Maniscalco]

On 8/27/24 9:48 PM, Akhil P Oommen wrote:
> On Fri, Aug 23, 2024 at 10:23:48AM +0100, Connor Abbott wrote:
>> On Fri, Aug 23, 2024 at 10:21 AM Connor Abbott <cwabbott0 at gmail.com> wrote:
>>>
>>> On Thu, Aug 22, 2024 at 9:06 PM Akhil P Oommen <quic_akhilpo at quicinc.com> wrote:
>>>>
>>>> On Wed, Aug 21, 2024 at 05:02:56PM +0100, Connor Abbott wrote:
>>>>> On Mon, Aug 19, 2024 at 9:09 PM Akhil P Oommen <quic_akhilpo at quicinc.com> wrote:
>>>>>>
>>>>>> On Thu, Aug 15, 2024 at 08:26:14PM +0200, Antonino Maniscalco wrote:
>>>>>>> This patch implements preemption feature for A6xx targets, this allows
>>>>>>> the GPU to switch to a higher priority ringbuffer if one is ready. A6XX
>>>>>>> hardware as such supports multiple levels of preemption granularities,
>>>>>>> ranging from coarse grained(ringbuffer level) to a more fine grained
>>>>>>> such as draw-call level or a bin boundary level preemption. This patch
>>>>>>> enables the basic preemption level, with more fine grained preemption
>>>>>>> support to follow.
>>>>>>>
>>>>>>> Signed-off-by: Sharat Masetty <smasetty at codeaurora.org>
>>>>>>> Signed-off-by: Antonino Maniscalco <antomani103 at gmail.com>
>>>>>>> ---
>>>>>>
>>>>>> No postamble packets which resets perfcounters? It is necessary. Also, I
>>>>>> think we should disable preemption during profiling like we disable slumber.
>>>>>>
>>>>>> -Akhil.
>>>>>
>>>>> I don't see anything in kgsl which disables preemption during
>>>>> profiling. It disables resetting perfcounters when doing system-wide
>>>>> profiling, like freedreno, and in that case I assume preempting is
>>>>> fine because the system profiler has a complete view of everything and
>>>>> should "see" preemptions through the traces. For something like
>>>>> VK_KHR_performance_query I suppose we'd want to disable preemption
>>>>> because we disable saving/restoring perf counters, but that has to
>>>>> happen in userspace because the kernel doesn't know what userspace
>>>>> does.
>>>>>
>>>>
>>>> KGSL does some sort of arbitration of perfcounter configurations and
>>>> adds the select/enablement reg configuration as part of dynamic
>>>> power up register list which we are not doing here. Is this something
>>>> you are taking care of from userspace via preamble?
>>>>
>>>> -Akhil
>>>
>>> I don't think we have to take care of that in userspace, because Mesa
>>> will always configure the counter registers before reading them in the
>>> same submission, and if it gets preempted in the meantime then we're
>>> toast anyways (due to not saving/restoring perf counters). kgsl sets
>>> them from userspace, which is why it has to do something to set them
>>
>> Sorry, should be "kgsl sets them from the kernel".
>>
>>> after IFPC slumber or a context switch when the HW state is gone.
>>> Also, because the upstream approach doesn't play nicely with system
>>> profilers like perfetto, VK_KHR_performance_query is hidden by default
>>> behind a debug flag in turnip. So there's already an element of "this
>>> is unsupported, you have to know what you're doing to use it."
> 
> But when you have composition on GPU enabled, there will be very frequent
> preemption. And I don't know how usable profiling tools will be in that
> case unless you disable preemption with a Mesa debug flag. But for that
> to work, all existing submitqueues should be destroyed and recreated.
> 
> So I was thinking that we can use the sysprof propertry to force L0
> preemption from kernel.
> 
> -Akhil.
> 

Right but when using a system profiler I imagined the expectation would 
be to be able to understand how applications and compositor interact. An 
use case could be measuring latency and understanding what contributes 
to it. That is actually the main reason I added traces for preemption. 
Disabling preemption would make it less useful for this type of 
analysis. Did you have an use case in mind for a system profiler that 
would benefit from disabling preemption and that is not covered by 
VK_KHR_performance_query (or equivalent GL ext)?

Best regards,
-- 
Antonino Maniscalco <antomani103 at gmail.com>