[igt-dev] [PATCH i-g-t] tests/psr: Add PSR tests to fast-feedback list.

Tue Jul 17 07:52:07 UTC 2018

On 17/07/18 08:13, Rodrigo Vivi wrote:
> On Mon, Jul 16, 2018 at 11:29:48AM -0700, Dhinakaran Pandiyan wrote:
>> On Mon, 2018-07-16 at 10:43 -0700, Rodrigo Vivi wrote:
>>> On Fri, Jul 13, 2018 at 04:39:04PM -0700, Dhinakaran Pandiyan wrote:
>>>>
>>>> We do not have PSR panels on shards machines, so this is the only
>>>> way to
>>>> test PSR in CI.
>>> We definitely need this...
>>> but we need to ckeck the overall time and ff limits here as well...
>>
>>  From https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1588/fi-kbl-r/runt
>> imes0.log
>>  19.46 igt at kms_psr@suspend pass
>>   4.78 igt at kms_psr@dpms pass
>>   3.37 igt at kms_psr@sprite_blt pass
>>   3.35 igt at kms_psr@sprite_plane_onoff pass
>>   3.35 igt at kms_psr@sprite_plane_move pass
>>   3.35 igt at kms_psr@sprite_mmap_gtt pass
>>   3.35 igt at kms_psr@sprite_mmap_cpu pass
>>   3.34 igt at kms_psr@sprite_render pass
>>   3.34 igt at kms_psr@primary_render pass
>>   3.34 igt at kms_psr@primary_page_flip pass
>>   3.33 igt at kms_psr@primary_mmap_cpu pass
>>   3.31 igt at kms_psr@primary_mmap_gtt pass
>>   3.31 igt at kms_psr@primary_blt pass
>>   3.29 igt at kms_psr@cursor_render pass
>>   3.29 igt at kms_psr@cursor_plane_onoff pass
>>   3.29 igt at kms_psr@cursor_plane_move pass
>>   3.29 igt at kms_psr@cursor_mmap_gtt pass
>>   3.29 igt at kms_psr@cursor_mmap_cpu pass
>>   3.29 igt at kms_psr@cursor_blt pass
>>   3.14 igt at kms_psr@basic pass
>>   3.11 igt at kms_psr@no_drrs pass
>>
>> Each test is under the 30s limit and kms_psr at suspend is the only that
>> gets closer to the limit.
>>
>> Without kms_psr subtests 730.18s TOTAL
>> https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_4488/fi-kbl-r/runtimes0
>> .log
>> With kms_psr subtests 626.50s TOTAL
>> https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1588/fi-kbl-r/runtimes0.
>> log
>>
>>> ditto on the kms_frontbuffer_tracking one...
>>
>> From https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1590/fi-whl-u/runti
>> mes0.log
>>
>>   7.38 igt at kms_frontbuffer_tracking@psr-1p-offscren-pri-shrfb-draw-pwri
>> te pass
>>   6.23 igt at kms_frontbuffer_tracking@fbcpsr-1p-indfb-fliptrack pass
>>   5.79 igt at kms_frontbuffer_tracking@psr-1p-primscrn-pri-indfb-draw-mmap
>> -cpu pass
>>   5.71 igt at kms_frontbuffer_tracking@psr-1p-primscrn-pri-shrfb-draw-mmap
>> -cpu pass
>>   5.69 igt at kms_frontbuffer_tracking@psr-1p-primscrn-pri-shrfb-draw-blt
>> pass
>>   5.53 igt at kms_frontbuffer_tracking@psr-farfromfence pass
>>   5.51 igt at kms_frontbuffer_tracking@fbcpsr-farfromfence pass
>>   5.48 igt at kms_frontbuffer_tracking@fbcpsr-1p-offscren-pri-indfb-draw-m
>> map-wc pass
>> 5.20 igt at kms_frontbuffer_tracking@psr-1p-primscrn-pri-indfb-draw-mmap-g
>> tt pass
>>   5.19 igt at kms_frontbuffer_tracking@psr-1p-primscrn-cur-indfb-draw-mmap
>> -wc pass
>>   5.18 igt at kms_frontbuffer_tracking@psr-rgb101010-draw-mmap-gtt pass
>>   5.18 igt at kms_frontbuffer_tracking@psr-1p-primscrn-pri-shrfb-draw-mmap
>> -gtt pass
>>   5.16 igt at kms_frontbuffer_tracking@fbcpsr-1p-primscrn-pri-indfb-draw-m
>> map-gtt pass
>>   5.14 igt at gem_sync@basic-each pass
>>   5.10 igt at kms_frontbuffer_tracking@fbcpsr-rgb565-draw-mmap-gtt pass
>>   5.08 igt at kms_frontbuffer_tracking@psr-rgb565-draw-mmap-gtt pass
>>   4.96 igt at kms_frontbuffer_tracking@psr-1p-offscren-pri-shrfb-draw-mmap
>> -gtt pass
>>   4.74 igt at kms_frontbuffer_tracking@psr-1p-primscrn-spr-indfb-draw-mmap
>> -gtt pass
>>   4.73 igt at kms_frontbuffer_tracking@fbcpsr-1p-primscrn-spr-indfb-draw-m
>> map-gtt pass
>>   4.73 igt at kms_frontbuffer_tracking@fbcpsr-1p-primscrn-cur-indfb-draw-m
>> map-gtt pass
>>   4.72 igt at kms_frontbuffer_tracking@psr-1p-primscrn-cur-indfb-draw-mmap
>> -gtt pass
>>   4.62 igt at kms_frontbuffer_tracking@fbcpsr-1p-offscren-pri-shrfb-draw-m
>> map-gtt skip
>>   4.61 igt at kms_frontbuffer_tracking@fbcpsr-1p-shrfb-fliptrack skip
>>   4.58 igt at kms_frontbuffer_tracking@fbcpsr-1p-primscrn-pri-shrfb-draw-m
>> map-gtt skip
>>   4.57 igt at kms_frontbuffer_tracking@fbcpsr-1p-primscrn-pri-shrfb-draw-r
>> ender skip
>>   3.63 igt at kms_frontbuffer_tracking@fbcpsr-stridechange skip
>>
>> Same here, all are under the individual test limit.
> 
> both changes
> 
> Acked-by: Rodrigo Vivi <rodrigo.vivi at intel.com>
> 
> unless CI folks nack it ;)

Well, no, that's can't be accepted as such. The point is not for tests
to take less than 30s, it is to have the widest coverage in less than 10m...

As a general rule, we are OK with an increase by 15 to 20s in the
general case, and no more than 30s on the slowest machine, but we can't
allow anything more than that in *fast* feedback.

In my opinion, the basic test should be the only one executed, but of
course you know better and may want to add one or two extra tests. The
suspend test should however definitely not be there because it increases
the chances of the machine dying for unrelated reasons :s

I know you are trying to add all these tests because right now, this is
the only way to prevent PSR regressions (since we have no PSR panels in
shards), but we just received enough machines to add to the shards that
have PSR panels and will catch regressions.

The point of FF should remain to gate further testing, when a
patchseries is just too broken to warrant wasting time on it ;)

Martin
> 
>>
>> I am not sure what the overall time limit is, this is what I see - 
>> Without PSR frontbuffer 742.12s
>> (https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_4498/fi-skl-6600u/runt
>> imes0.log)
>> With PSR frontbuffer tests 693.63s
>> (https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1590/fi-skl-6600u/runti
>> mes0.log)
>>
>> Looks like there are others tests with a larger variance negating the
>> impact of new PSR tests.
>>
>> -DK
>>
>>>
>>>>
>>>>
>>>> Cc: Rodrigo Vivi <rodrigo.vivi at intel.com>
>>>> Cc: Petri Latvala <petri.latvala at intel.com>
>>>> Signed-off-by: Dhinakaran Pandiyan <dhinakaran.pandiyan at intel.com>
>>>> ---
>>>>  tests/intel-ci/fast-feedback.testlist | 20 ++++++++++++++++++++
>>>>  1 file changed, 20 insertions(+)
>>>>
>>>> diff --git a/tests/intel-ci/fast-feedback.testlist b/tests/intel-
>>>> ci/fast-feedback.testlist
>>>> index aa861d39..e262d024 100644
>>>> --- a/tests/intel-ci/fast-feedback.testlist
>>>> +++ b/tests/intel-ci/fast-feedback.testlist
>>>> @@ -244,6 +244,26 @@ igt at kms_pipe_crc_basic@suspend-read-crc-pipe-a
>>>>  igt at kms_pipe_crc_basic@suspend-read-crc-pipe-b
>>>>  igt at kms_pipe_crc_basic@suspend-read-crc-pipe-c
>>>>  igt at kms_psr@basic
>>>> +igt at kms_psr@no_drrs
>>>> +igt at kms_psr@primary_page_flip
>>>> +igt at kms_psr@primary_mmap_gtt
>>>> +igt at kms_psr@primary_mmap_cpu
>>>> +igt at kms_psr@primary_blt
>>>> +igt at kms_psr@primary_render
>>>> +igt at kms_psr@sprite_mmap_gtt
>>>> +igt at kms_psr@sprite_mmap_cpu
>>>> +igt at kms_psr@sprite_blt
>>>> +igt at kms_psr@sprite_render
>>>> +igt at kms_psr@sprite_plane_move
>>>> +igt at kms_psr@sprite_plane_onoff
>>>> +igt at kms_psr@cursor_mmap_gtt
>>>> +igt at kms_psr@cursor_mmap_cpu
>>>> +igt at kms_psr@cursor_blt
>>>> +igt at kms_psr@cursor_render
>>>> +igt at kms_psr@cursor_plane_move
>>>> +igt at kms_psr@cursor_plane_onoff
>>>> +igt at kms_psr@dpms
>>>> +igt at kms_psr@suspend
>>>>  igt at kms_setmode@basic-clone-single-crtc
>>>>  igt at kms_sink_crc_basic
>>>>  igt at pm_backlight@basic-brightness
>> _______________________________________________
>> igt-dev mailing list
>> igt-dev at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/igt-dev