[igt-dev] [PATCH i-g-t] blacklist: Don't run DRRS test on Intel CI system

[Bs, Lohith]

On 3/27/2018 5:05 PM, Saarinen, Jani wrote:
>
>> -----Original Message-----
>> From: igt-dev [mailto:igt-dev-bounces at lists.freedesktop.org] On Behalf Of
>> Lofstedt, Marta
>> Sent: tiistai 27. maaliskuuta 2018 14.05
>> To: Latvala, Petri <petri.latvala at intel.com>; Hiler, Arkadiusz
>> <arkadiusz.hiler at intel.com>
>> Cc: igt-dev at lists.freedesktop.org; Daniel Vetter <daniel at ffwll.ch>
>> Subject: Re: [igt-dev] [PATCH i-g-t] blacklist: Don't run DRRS test on Intel CI
>> system
>>
>>
>>
>>> -----Original Message-----
>>> From: Latvala, Petri
>>> Sent: Tuesday, March 27, 2018 2:02 PM
>>> To: Hiler, Arkadiusz <arkadiusz.hiler at intel.com>
>>> Cc: Lofstedt, Marta <marta.lofstedt at intel.com>; Maarten Lankhorst
>>> <maarten.lankhorst at linux.intel.com>; igt-dev at lists.freedesktop.org;
>>> Chris Wilson <chris at chris-wilson.co.uk>; Daniel Vetter
>>> <daniel at ffwll.ch>
>>> Subject: Re: [igt-dev] [PATCH i-g-t] blacklist: Don't run DRRS test on
>>> Intel CI system
>>>
>>> On Tue, Mar 27, 2018 at 01:50:10PM +0300, Arkadiusz Hiler wrote:
>>>> On Tue, Mar 27, 2018 at 01:06:16PM +0300, Lofstedt, Marta wrote:
>>>>>> -----Original Message-----
>>>>>> From: Hiler, Arkadiusz
>>>>>> Sent: Tuesday, March 27, 2018 12:15 PM
>>>>>> To: Maarten Lankhorst <maarten.lankhorst at linux.intel.com>
>>>>>> Cc: Lofstedt, Marta <marta.lofstedt at intel.com>; igt-
>>>>>> dev at lists.freedesktop.org; Chris Wilson
>>>>>> <chris at chris-wilson.co.uk>; Latvala, Petri
>>>>>> <petri.latvala at intel.com>; Daniel Vetter <daniel at ffwll.ch>
>>>>>> Subject: Re: [igt-dev] [PATCH i-g-t] blacklist: Don't run DRRS
>>>>>> test on Intel CI system
>>>>>>
>>>>>> On Mon, Mar 26, 2018 at 03:22:56PM +0200, Maarten Lankhorst wrote:
>>>>>>> Op 26-03-18 om 08:29 schreef Marta Lofstedt:
>>>>>>>> Skipping takes time, specifically for the big amount of drrs
>>>>>>>> related kms_frontbuffer_tracking tests. Since we currently
>>>>>>>> don't have any system set up with DRRS panels, blacklisting
>>>>>>>> all those test will save time, and we can avoid the need to
>>>>>>>> increase the Jenkins timeout in order to solve the Bugzilla below.
>>>>>>>>
>>>>>>>> Bugzilla:
>>>>>>>> https://bugs.freedesktop.org/show_bug.cgi?id=105617
>>>>>>>> Signed-off-by: Marta Lofstedt <marta.lofstedt at intel.com>
>>>>>>>> ---
>>>>>>>>   tests/intel-ci/blacklist.txt | 1 +
>>>>>>>>   1 file changed, 1 insertion(+)
>>>>>>>>
>>>>>>>> diff --git a/tests/intel-ci/blacklist.txt
>>>>>>>> b/tests/intel-ci/blacklist.txt index 7ca313ac..0a8506ca
>>>>>>>> 100644
>>>>>>>> --- a/tests/intel-ci/blacklist.txt
>>>>>>>> +++ b/tests/intel-ci/blacklist.txt
>>>>>>>> @@ -57,6 +57,7 @@ igt at gem_tiled_swapping@(?!non-
>>> threaded).*
>>> igt at gem_userptr_blits@(major|minor|forked|mlocked|swapping).*
>>>>>>>>   igt at gem_wait@.*hang.*
>>>>>>>>   igt at gem_write_read_ring_switch(@.*)?
>>>>>>>> +igt at kms_frontbuffer_tracking@.*drrs*
>>>>>> I think we should start annotating those, so the reevaluation
>>>>>> would be easier than 'git blame'-ing the file and going through
>>>>>> each
>>> commit.
>>>>>>>>   ###############################################
>>>>>>>>   # Broadcom
>>>>>>>>   ###############################################
>>>>>>> Hm a single skipped test should take about 100 ms, but I do
>>>>>>> notice we probe all outputs in igt_display_init, so could this
>>>>>>> also be fixed by https://patchwork.freedesktop.org/patch/212983/ ?
>>>>>>>
>>>>>>> This should reduce probing to the minimum.
>>>>>> Note the discussion around this patch:
>>>>>> http://patchwork.freedesktop.org/patch/204944/
>>>>>>
>>>>>> The idea was to start async probe during the boot, so we won't
>>>>>> depend on the fbdev to do the initial probing.
>>>>>>
>>>>>> But it seems that, after all, UNKNOWN is still a thing. I should
>>>>>> have checked myself...
>>>>>>
>>>>>> I've pulled the numbers for your change:
>>>>>>
>>>>>>               CI_DRM_3978      IGTPW_1192
>>>>>> shard-apl         12982s          12783s
>>>>>> shard-hsw         11735s          11560s
>>>>>> shard-snb          7033s           6906s
>>>>>> shard-kbl          9843s           9048s
>>>>>>
>>>>> Thanks for numbers Arek.
>>>>>
>>>>> However, my main motivation for doing this is to avoid increasing
>>>>> 18 minutes Jenkins timeout in bug:
>>>>> https://bugs.freedesktop.org/show_bug.cgi?id=105617
>>>> Even with Marteen's change the drrs tests take
>>>> 570 * ~0.185s = ~100s to skip.
>>>>
>>>> Also I think you've meant:
>>>> +igt at kms_frontbuffer_tracking@.*drrs.*
>>>> instead of:
>>>> +igt at kms_frontbuffer_tracking@.*drrs*
>>>
>>> Yep, this.
>>>
>>>
>>>
>>>> I am fine with getting the change in, for now.
>>> I commented on IRC earlier, but let's put it in the mailing list also.
>>>
>>> This patch needs:
>>>
>>> 1) Above typo fixed
>>> 2) Comment explaining why it's blacklisted
>>> 3) Acks from developers working on DRRS, to confirm they know CI won't
>>> be testing their code.
>> Do we have Acks from all other developers that already have tests on the
>> blacklist because they take too long to execute on our CI system?
> + Lohith to comment.
Hi Jani,
We accept to blacklist 570 DRRS subtests under kms_frontbuffer_tracking 
as we don't
have DRRS supported panel.

I tried to check the time it takes for running 570 DRRS subtests on GLK 
silicon having
non-DRRS edp panel + HDMI. It took "2m 13.834s".
>>>
>>>
>>> --
>>> Petri Latvala