[Intel-gfx] About dealing with CSB.context element switch in execlist mode.

[Dave Gordon]

On 25/11/15 13:17, Wang, Zhi A wrote:
> OK. I see. Thanks Michel! :) Have a nice day. :)
>
> Thanks,
> Zhi.
>
>> -----Original Message-----
>> From: Thierry, Michel
>> Sent: Wednesday, November 25, 2015 9:15 PM
>> To: Wang, Zhi A; intel-gfx at lists.freedesktop.org
>> Cc: Han, Xu; Li, Weinan Z; He, Min; Lv, Zhiyuan; Tian, Kevin
>> Subject: Re: [Intel-gfx] About dealing with CSB.context element switch in
>> execlist mode.
>>
>> On 11/25/2015 1:00 PM, Wang, Zhi A wrote:
>>> Another question about EXECLIST is: Can a preemption happen between
>> element switch?
>>>
>>> I know this is beyond the scope of i915 a little. I'm just curious if it's possible.
>>>
>>> Let's say we have context A B C
>>>
>>> At first, we submit context A B in one ELSP write.
>>> Then, we submit context C in another ELSP write at some time.
>>>
>>> If context A or B is running and gets preempted, then there will be
>> CSB.preempted in CSB buffer. This is the normal behavior.
>>>
>>> I'm wondering that if there is any possibility that a preemption can happen
>> between the two elements.
>>>
>>> Then the CSB should look like:
>>>
>>> [CSB 0 idle-to-active]
>>> [CSB 1 CTX A element_switch/context_complete] [CSB 2 CTX C
>>> active-to-idle/context_complete]
>>>
>>> Is it possible?
>>
>> I would expect to always have a preempted event in the CSB, even in the
>> remote case that A already completed and B hasn't started; there was an
>> active execlist and it has been replaced by a new one:
>>
>>    [CSB 0 idle-to-active]
>>    [CSB 1 CTX A element_switch/context_complete]
>>    [CSB 2 preempted ]
>>    [CSB 3 CTX C active-to-idle/context_complete]
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/intel-gfx

A few examples of CSB sequences you may see:

CSB[1]: 0x000.00000 / 0x00000001  I->A
CSB[2]: 0x201.018df / 0x00000018                 A->I DONE
CSB[3]: 0x000.00000 / 0x00000001  I->A
CSB[4]: 0x201.018df / 0x00000018                 A->I DONE
CSB[5]: 0x000.00000 / 0x00000001  I->A
CSB[0]: 0x201.018df / 0x00000018                 A->I DONE

This is as simple as it gets. A single context is repeatedly submitted, 
and each time it completes before being resubmitted.

CSB[3]: 0x000.00000 / 0x00000001  I->A
CSB[4]: 0x601.018e6 / 0x00008002       PRMT                LITE
CSB[5]: 0x601.018e6 / 0x00000018                 A->I DONE
CSB[0]: 0x000.00000 / 0x00000001  I->A
CSB[1]: 0x601.018e6 / 0x00008002       PRMT                LITE
CSB[2]: 0x601.018e6 / 0x00000018                 A->I DONE

Here again a single context is in use, but this time it was submitted, 
then updated and resubmitted while still executing (lite restore), then 
completed. This whole sequence repeats twice.

CSB[4]: 0x001.02935 / 0x00000018                 A->I DONE
CSB[5]: 0x000.00000 / 0x00000001  I->A
CSB[0]: 0x001.02935 / 0x00000018                 A->I DONE
CSB[1]: 0x000.00000 / 0x00000001  I->A
CSB[2]: 0x001.02935 / 0x00000002       PRMT
CSB[3]: 0x001.02950 / 0x00000018                 A->I DONE

This is a genuine preemption. We see context A (02935) being submitted 
and completed twice, but during the third execution is gets preempted. 
The last line shows the completion of context B (02950) that preempted 
context A.

CSB[3]: 0x000.00000 / 0x00000001  I->A
CSB[4]: 0x001.018ed / 0x00000002       PRMT
CSB[5]: 0x001.01908 / 0x00000018                 A->I DONE
CSB[0]: 0x000.00000 / 0x00000001  I->A
CSB[1]: 0x001.01908 / 0x00000018                 A->I DONE
CSB[2]: 0x000.00000 / 0x00000001  I->A

Here, context A (018ed) was started, then preempted by a higher-priority 
context B (01908), which got to run to completion twice before another 
context (presumably A) was (re)started. We infer that A (or whatever 
other context was most recently submitted) was still running at the 
instant when this trace was captured.

The PRMT (without LITE) event always indicates the identity of the 
context that was preempted. In the case of preemption after a dual 
submission, it may be either the first or the second workload that gets 
preempted, but it must be one or the other (there's no "idle" window 
between finishing the first and starting the second in which preemption 
could occur). In addition, preemption is actually disabled while context 
restore is in progress, so if the ELSP is written after the GPU has 
committed to loading a context, the preemption won't happen until the 
load is completed. [This is subject to change, but this how it's set up 
on Skylake today].

HTH,
.Dave.