[igt-dev] ✓ Fi.CI.IGT: success for tests/kms_frontbuffer_tracking: Rework to prevent modesets!

[Lofstedt, Marta]

> -----Original Message-----
> From: Vivi, Rodrigo
> Sent: Thursday, April 5, 2018 11:36 PM
> To: Daniel Vetter <daniel.vetter at ffwll.ch>
> Cc: IGT development <igt-dev at lists.freedesktop.org>; Sarvela, Tomi P
> <tomi.p.sarvela at intel.com>; Hiler, Arkadiusz <arkadiusz.hiler at intel.com>;
> Saarinen, Jani <jani.saarinen at intel.com>; Lofstedt, Marta
> <marta.lofstedt at intel.com>; Zanoni, Paulo R <paulo.r.zanoni at intel.com>;
> Pandiyan, Dhinakaran <dhinakaran.pandiyan at intel.com>; Maarten
> Lankhorst <maarten.lankhorst at linux.intel.com>
> Subject: Re: [igt-dev] ✓ Fi.CI.IGT: success for tests/kms_frontbuffer_tracking:
> Rework to prevent modesets!
> 
> On Thu, Apr 05, 2018 at 09:56:42PM +0200, Daniel Vetter wrote:
> > Hi all,
> >
> > Ok, something went wrong here it seems:
> >
> > This patch series passed CI nicely, but then it caused a regression:
> >
> > https://bugs.freedesktop.org/show_bug.cgi?id=105503
> >
> > Another problem is that history moves so damn fast nowadays that even
> > the longer per-test history is useless already:
> >
> > https://intel-gfx-ci.01.org/tree/drm-tip/igt@kms_frontbuffer_tracking@
> > fbc-rgb101010-draw-mmap-cpu.html
> >
> > Just 1 week later I can't even see anymore where this started. This
> > makes finding and analysing regressions a real pain.
> >
> > Tomi/Arek, can we have more history here? Or some view to go back
> further.
> 
> There is an extra aspect here. Paulo raised that the removal of modesets
> could fully invalidate the kms_frontbuffer_tracking entirely.
> 
> So I think we should revert this patch.
> 
For this bug I don't think we need to revert the patch, I believe it could be fixed by having the badformat_subtest behave more like the draw_subtest. 

However, I am really concerned with https://bugs.freedesktop.org/show_bug.cgi?id=105798 
On SNB-shard and IVB drmtip runs. I have noticed that kms tests that are always skipped now can fail on random shards on random machines. This appear to always start with a kms_frontbuffer_tracking failing then other kms test can be affected. It appear as if this sorts itself out after for example a kms_flip test was run. Since the frequency of hitting this only is ~10% and it starts at random tests it is very difficult to say when it started. My theory is that some recent frontbuffer tracking rework don't take into account first frontbuffer_tracking test is supposed to be skipped, 

> >
> > Cheers, Daniel
> >
> >
> > On Tue, Mar 20, 2018 at 12:46 PM, Patchwork
> > <patchwork at emeril.freedesktop.org> wrote:
> > > == Series Details ==
> > >
> > > Series: tests/kms_frontbuffer_tracking: Rework to prevent modesets!
> > > URL   : https://patchwork.freedesktop.org/series/40195/
> > > State : success
> > >
> > > == Summary ==
> > >
> > > ---- Possible new issues:
> > >
> > > Test kms_cursor_crc:
> > >         Subgroup cursor-128x128-suspend:
> > >                 skip       -> PASS       (shard-snb)
> > >
> > > ---- Known issues:
> > >
> > > Test kms_cursor_legacy:
> > >         Subgroup 2x-long-flip-vs-cursor-legacy:
> > >                 incomplete -> PASS       (shard-hsw) fdo#104873
> > > Test kms_flip:
> > >         Subgroup dpms-vs-vblank-race:
> > >                 fail       -> PASS       (shard-hsw) fdo#103060
> > >         Subgroup flip-vs-absolute-wf_vblank:
> > >                 fail       -> PASS       (shard-hsw) fdo#100368
> > > Test kms_flip_tiling:
> > >         Subgroup flip-x-tiled:
> > >                 fail       -> PASS       (shard-apl) fdo#103822
> > > Test kms_pipe_crc_basic:
> > >         Subgroup suspend-read-crc-pipe-a:
> > >                 skip       -> PASS       (shard-snb) fdo#103375 +1
> > > Test kms_plane_multiple:
> > >         Subgroup atomic-pipe-a-tiling-x:
> > >                 pass       -> FAIL       (shard-snb) fdo#103166 +1
> > >
> > > fdo#104873 https://bugs.freedesktop.org/show_bug.cgi?id=104873
> > > fdo#103060 https://bugs.freedesktop.org/show_bug.cgi?id=103060
> > > fdo#100368 https://bugs.freedesktop.org/show_bug.cgi?id=100368
> > > fdo#103822 https://bugs.freedesktop.org/show_bug.cgi?id=103822
> > > fdo#103375 https://bugs.freedesktop.org/show_bug.cgi?id=103375
> > > fdo#103166 https://bugs.freedesktop.org/show_bug.cgi?id=103166
> > >
> > > shard-apl        total:3478 pass:1814 dwarn:1   dfail:0   fail:7   skip:1655
> time:13117s
> > > shard-hsw        total:3478 pass:1768 dwarn:1   dfail:0   fail:1   skip:1707
> time:11755s
> > > shard-snb        total:3478 pass:1357 dwarn:1   dfail:0   fail:3   skip:2117
> time:7228s
> > > Blacklisted hosts:
> > > shard-kbl        total:3478 pass:1937 dwarn:2   dfail:1   fail:9   skip:1529
> time:9827s
> > >
> > > == Logs ==
> > >
> > > For more details see:
> > > https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1166/shards.html
> > > _______________________________________________
> > > igt-dev mailing list
> > > igt-dev at lists.freedesktop.org
> > > https://lists.freedesktop.org/mailman/listinfo/igt-dev
> >
> >
> >
> > --
> > Daniel Vetter
> > Software Engineer, Intel Corporation
> > +41 (0) 79 365 57 48 - http://blog.ffwll.ch