[igt-dev] [PATCH i-g-t] blacklist: Don't run DRRS test on Intel CI system

[Petri Latvala]

On Tue, Mar 27, 2018 at 02:05:28PM +0300, Lofstedt, Marta wrote:
> 
> 
> > -----Original Message-----
> > From: Latvala, Petri
> > Sent: Tuesday, March 27, 2018 2:02 PM
> > To: Hiler, Arkadiusz <arkadiusz.hiler at intel.com>
> > Cc: Lofstedt, Marta <marta.lofstedt at intel.com>; Maarten Lankhorst
> > <maarten.lankhorst at linux.intel.com>; igt-dev at lists.freedesktop.org; Chris
> > Wilson <chris at chris-wilson.co.uk>; Daniel Vetter <daniel at ffwll.ch>
> > Subject: Re: [igt-dev] [PATCH i-g-t] blacklist: Don't run DRRS test on Intel CI
> > system
> > 
> > On Tue, Mar 27, 2018 at 01:50:10PM +0300, Arkadiusz Hiler wrote:
> > > On Tue, Mar 27, 2018 at 01:06:16PM +0300, Lofstedt, Marta wrote:
> > > > > -----Original Message-----
> > > > > From: Hiler, Arkadiusz
> > > > > Sent: Tuesday, March 27, 2018 12:15 PM
> > > > > To: Maarten Lankhorst <maarten.lankhorst at linux.intel.com>
> > > > > Cc: Lofstedt, Marta <marta.lofstedt at intel.com>; igt-
> > > > > dev at lists.freedesktop.org; Chris Wilson
> > > > > <chris at chris-wilson.co.uk>; Latvala, Petri
> > > > > <petri.latvala at intel.com>; Daniel Vetter <daniel at ffwll.ch>
> > > > > Subject: Re: [igt-dev] [PATCH i-g-t] blacklist: Don't run DRRS
> > > > > test on Intel CI system
> > > > >
> > > > > On Mon, Mar 26, 2018 at 03:22:56PM +0200, Maarten Lankhorst wrote:
> > > > > > Op 26-03-18 om 08:29 schreef Marta Lofstedt:
> > > > > > > Skipping takes time, specifically for the big amount of drrs
> > > > > > > related kms_frontbuffer_tracking tests. Since we currently
> > > > > > > don't have any system set up with DRRS panels, blacklisting
> > > > > > > all those test will save time, and we can avoid the need to
> > > > > > > increase the Jenkins timeout in order to solve the Bugzilla below.
> > > > > > >
> > > > > > > Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=105617
> > > > > > > Signed-off-by: Marta Lofstedt <marta.lofstedt at intel.com>
> > > > > > > ---
> > > > > > >  tests/intel-ci/blacklist.txt | 1 +
> > > > > > >  1 file changed, 1 insertion(+)
> > > > > > >
> > > > > > > diff --git a/tests/intel-ci/blacklist.txt
> > > > > > > b/tests/intel-ci/blacklist.txt index 7ca313ac..0a8506ca 100644
> > > > > > > --- a/tests/intel-ci/blacklist.txt
> > > > > > > +++ b/tests/intel-ci/blacklist.txt
> > > > > > > @@ -57,6 +57,7 @@ igt at gem_tiled_swapping@(?!non-
> > threaded).*
> > > > > > >
> > igt at gem_userptr_blits@(major|minor|forked|mlocked|swapping).*
> > > > > > >  igt at gem_wait@.*hang.*
> > > > > > >  igt at gem_write_read_ring_switch(@.*)?
> > > > > > > +igt at kms_frontbuffer_tracking@.*drrs*
> > > > >
> > > > > I think we should start annotating those, so the reevaluation
> > > > > would be easier than 'git blame'-ing the file and going through each
> > commit.
> > > > >
> > > > > > >  ###############################################
> > > > > > >  # Broadcom
> > > > > > >  ###############################################
> > > > > > Hm a single skipped test should take about 100 ms, but I do
> > > > > > notice we probe all outputs in igt_display_init, so could this
> > > > > > also be fixed by https://patchwork.freedesktop.org/patch/212983/ ?
> > > > > >
> > > > > > This should reduce probing to the minimum.
> > > > >
> > > > > Note the discussion around this patch:
> > > > > http://patchwork.freedesktop.org/patch/204944/
> > > > >
> > > > > The idea was to start async probe during the boot, so we won't
> > > > > depend on the fbdev to do the initial probing.
> > > > >
> > > > > But it seems that, after all, UNKNOWN is still a thing. I should
> > > > > have checked myself...
> > > > >
> > > > > I've pulled the numbers for your change:
> > > > >
> > > > >              CI_DRM_3978      IGTPW_1192
> > > > > shard-apl         12982s          12783s
> > > > > shard-hsw         11735s          11560s
> > > > > shard-snb          7033s           6906s
> > > > > shard-kbl          9843s           9048s
> > > > >
> > > > Thanks for numbers Arek.
> > > >
> > > > However, my main motivation for doing this is to avoid increasing 18
> > > > minutes Jenkins timeout in bug:
> > > > https://bugs.freedesktop.org/show_bug.cgi?id=105617
> > >
> > > Even with Marteen's change the drrs tests take
> > > 570 * ~0.185s = ~100s to skip.
> > >
> > > Also I think you've meant:
> > > +igt at kms_frontbuffer_tracking@.*drrs.*
> > > instead of:
> > > +igt at kms_frontbuffer_tracking@.*drrs*
> > 
> > 
> > Yep, this.
> > 
> > 
> > 
> > >
> > > I am fine with getting the change in, for now.
> > 
> > I commented on IRC earlier, but let's put it in the mailing list also.
> > 
> > This patch needs:
> > 
> > 1) Above typo fixed
> > 2) Comment explaining why it's blacklisted
> > 3) Acks from developers working on DRRS, to confirm they know CI won't be
> > testing their code.
> Do we have Acks from all other developers that already have tests on the blacklist because they take too long to execute on our CI system?


We got the original blacklist of GEM tests from the GEM team.



-- 
Petri Latvala



> 
> > 
> > 
> > 
> > --
> > Petri Latvala