[igt-dev] Intel CI blacklist suggestion for tests that are skipped on all shards

[Jani Nikula]

On Wed, 07 Feb 2018, "Lofstedt, Marta" <marta.lofstedt at intel.com> wrote:
>> -----Original Message-----
>> From: Jani Nikula [mailto:jani.nikula at linux.intel.com]
>> Sent: Tuesday, February 6, 2018 8:42 PM
>> To: Lofstedt, Marta <marta.lofstedt at intel.com>; igt-
>> dev at lists.freedesktop.org
>> Cc: Sarvela, Tomi P <tomi.p.sarvela at intel.com>; Peres, Martin
>> <martin.peres at intel.com>
>> Subject: Re: [igt-dev] Intel CI blacklist suggestion for tests that are skipped on
>> all shards
>> 
>> On Tue, 06 Feb 2018, "Lofstedt, Marta" <marta.lofstedt at intel.com> wrote:
>> > After summarizing the skip time for our shards:
>> > GLK: 428s, APL: 371s, SNB:325s, KBL:283, HSW:182s
>> 
>> Why does it take so long on any of those platforms to skip the tests?
>
> I believe it is the huge amount of skipped tests that is the main
> culprit, ~1500*0,1 seconds is real time. And the new DRRS test
> contribute a lot.
>
> Also, I summarized some timings of the the kms_frontbuffer_tracking on
> GLK-shards:
>
> Total time = 2153s, note this is more than half of the total GLK-shard
> runtime.  Skipped time = 238s, this has more than doubled due to new
> DRRS tests

> Fail time = 1365s, this is mostly for the PSR tests that have been
> failing for months, due to unsupported panels being shipped with GLK
> RVPs. Unfortunately we don't have a full set of supported PSR
> panels. And if we changed to supported PSR panels I assume that the
> runtime would only increase further.

Okay. I'm just wondering about the possibilities of optimizing the
skipping in the tests themselves, perhaps by bailing out earlier,
vs. maintainability of a single big blacklist which combines plenty of
features across plenty of platforms.

BR,
Jani.


>
>
>> 
>> BR,
>> Jani.
>> 
>> 
>> --
>> Jani Nikula, Intel Open Source Technology Center

-- 
Jani Nikula, Intel Open Source Technology Center