[igt-dev] [PATCH i-g-t] blacklist: Don't run DRRS test on Intel CI system

[Saarinen, Jani]

> -----Original Message-----
> From: igt-dev [mailto:igt-dev-bounces at lists.freedesktop.org] On Behalf Of
> Lofstedt, Marta
> Sent: tiistai 27. maaliskuuta 2018 14.05
> To: Latvala, Petri <petri.latvala at intel.com>; Hiler, Arkadiusz
> <arkadiusz.hiler at intel.com>
> Cc: igt-dev at lists.freedesktop.org; Daniel Vetter <daniel at ffwll.ch>
> Subject: Re: [igt-dev] [PATCH i-g-t] blacklist: Don't run DRRS test on Intel CI
> system
> 
> 
> 
> > -----Original Message-----
> > From: Latvala, Petri
> > Sent: Tuesday, March 27, 2018 2:02 PM
> > To: Hiler, Arkadiusz <arkadiusz.hiler at intel.com>
> > Cc: Lofstedt, Marta <marta.lofstedt at intel.com>; Maarten Lankhorst
> > <maarten.lankhorst at linux.intel.com>; igt-dev at lists.freedesktop.org;
> > Chris Wilson <chris at chris-wilson.co.uk>; Daniel Vetter
> > <daniel at ffwll.ch>
> > Subject: Re: [igt-dev] [PATCH i-g-t] blacklist: Don't run DRRS test on
> > Intel CI system
> >
> > On Tue, Mar 27, 2018 at 01:50:10PM +0300, Arkadiusz Hiler wrote:
> > > On Tue, Mar 27, 2018 at 01:06:16PM +0300, Lofstedt, Marta wrote:
> > > > > -----Original Message-----
> > > > > From: Hiler, Arkadiusz
> > > > > Sent: Tuesday, March 27, 2018 12:15 PM
> > > > > To: Maarten Lankhorst <maarten.lankhorst at linux.intel.com>
> > > > > Cc: Lofstedt, Marta <marta.lofstedt at intel.com>; igt-
> > > > > dev at lists.freedesktop.org; Chris Wilson
> > > > > <chris at chris-wilson.co.uk>; Latvala, Petri
> > > > > <petri.latvala at intel.com>; Daniel Vetter <daniel at ffwll.ch>
> > > > > Subject: Re: [igt-dev] [PATCH i-g-t] blacklist: Don't run DRRS
> > > > > test on Intel CI system
> > > > >
> > > > > On Mon, Mar 26, 2018 at 03:22:56PM +0200, Maarten Lankhorst wrote:
> > > > > > Op 26-03-18 om 08:29 schreef Marta Lofstedt:
> > > > > > > Skipping takes time, specifically for the big amount of drrs
> > > > > > > related kms_frontbuffer_tracking tests. Since we currently
> > > > > > > don't have any system set up with DRRS panels, blacklisting
> > > > > > > all those test will save time, and we can avoid the need to
> > > > > > > increase the Jenkins timeout in order to solve the Bugzilla below.
> > > > > > >
> > > > > > > Bugzilla:
> > > > > > > https://bugs.freedesktop.org/show_bug.cgi?id=105617
> > > > > > > Signed-off-by: Marta Lofstedt <marta.lofstedt at intel.com>
> > > > > > > ---
> > > > > > >  tests/intel-ci/blacklist.txt | 1 +
> > > > > > >  1 file changed, 1 insertion(+)
> > > > > > >
> > > > > > > diff --git a/tests/intel-ci/blacklist.txt
> > > > > > > b/tests/intel-ci/blacklist.txt index 7ca313ac..0a8506ca
> > > > > > > 100644
> > > > > > > --- a/tests/intel-ci/blacklist.txt
> > > > > > > +++ b/tests/intel-ci/blacklist.txt
> > > > > > > @@ -57,6 +57,7 @@ igt at gem_tiled_swapping@(?!non-
> > threaded).*
> > > > > > >
> > igt at gem_userptr_blits@(major|minor|forked|mlocked|swapping).*
> > > > > > >  igt at gem_wait@.*hang.*
> > > > > > >  igt at gem_write_read_ring_switch(@.*)?
> > > > > > > +igt at kms_frontbuffer_tracking@.*drrs*
> > > > >
> > > > > I think we should start annotating those, so the reevaluation
> > > > > would be easier than 'git blame'-ing the file and going through
> > > > > each
> > commit.
> > > > >
> > > > > > >  ###############################################
> > > > > > >  # Broadcom
> > > > > > >  ###############################################
> > > > > > Hm a single skipped test should take about 100 ms, but I do
> > > > > > notice we probe all outputs in igt_display_init, so could this
> > > > > > also be fixed by https://patchwork.freedesktop.org/patch/212983/ ?
> > > > > >
> > > > > > This should reduce probing to the minimum.
> > > > >
> > > > > Note the discussion around this patch:
> > > > > http://patchwork.freedesktop.org/patch/204944/
> > > > >
> > > > > The idea was to start async probe during the boot, so we won't
> > > > > depend on the fbdev to do the initial probing.
> > > > >
> > > > > But it seems that, after all, UNKNOWN is still a thing. I should
> > > > > have checked myself...
> > > > >
> > > > > I've pulled the numbers for your change:
> > > > >
> > > > >              CI_DRM_3978      IGTPW_1192
> > > > > shard-apl         12982s          12783s
> > > > > shard-hsw         11735s          11560s
> > > > > shard-snb          7033s           6906s
> > > > > shard-kbl          9843s           9048s
> > > > >
> > > > Thanks for numbers Arek.
> > > >
> > > > However, my main motivation for doing this is to avoid increasing
> > > > 18 minutes Jenkins timeout in bug:
> > > > https://bugs.freedesktop.org/show_bug.cgi?id=105617
> > >
> > > Even with Marteen's change the drrs tests take
> > > 570 * ~0.185s = ~100s to skip.
> > >
> > > Also I think you've meant:
> > > +igt at kms_frontbuffer_tracking@.*drrs.*
> > > instead of:
> > > +igt at kms_frontbuffer_tracking@.*drrs*
> >
> >
> > Yep, this.
> >
> >
> >
> > >
> > > I am fine with getting the change in, for now.
> >
> > I commented on IRC earlier, but let's put it in the mailing list also.
> >
> > This patch needs:
> >
> > 1) Above typo fixed
> > 2) Comment explaining why it's blacklisted
> > 3) Acks from developers working on DRRS, to confirm they know CI won't
> > be testing their code.
> Do we have Acks from all other developers that already have tests on the
> blacklist because they take too long to execute on our CI system?
+ Lohith to comment.
> 
> >
> >
> >
> > --
> > Petri Latvala