[Intel-gfx] [RFC i-g-t 0/4] Redundant test pruning

[Daniel Vetter]

On Wed, Jul 05, 2017 at 02:30:43PM +0100, Tvrtko Ursulin wrote:
> 
> On 27/06/2017 09:02, Tvrtko Ursulin wrote:
> > On 26/06/2017 17:09, Daniel Vetter wrote:
> > > On Fri, Jun 23, 2017 at 12:31:39PM +0100, Tvrtko Ursulin wrote:
> > > > From: Tvrtko Ursulin <tvrtko.ursulin at intel.com>
> > > > 
> > > > Small series which saves test execution time by removing the
> > > > redundant tests.
> > > > 
> > > > Tvrtko Ursulin (4):
> > > >    igt: Remove default from the engine list
> > > >    gem_exec_basic: Exercise the default engine selection
> > > >    gem_sync: Add all and store_all subtests
> > > >    extended.testlist: Remove some test-subtest combinations
> > > 
> > > Ack on patches 1&2, but I'm not sold on patch 3. Atm gem_* takes a
> > > ridiculous amount of machine time to run, you're adding more stuff. Are
> > > those tests really drastially better at catching races if we run them 10x
> > > longer? Is there no better way to exercise the races (lots more machines,
> > > maybe slower ones, which is atm impossible since it just takes way, way
> > > too long and we need an entire farm just for one machine).
> > 
> > New gem_sync subtests were suggested by Chris after I send the first
> > version of the series with the goal of getting the same coverage in
> > faster time.
> > 
> > If you look at patch 4, it removes 18 * 150s of gem_sync subtests, and
> > adds 4 * 150s. So in total we are 35 minutes better of in the best case,
> > a bit less on smaller machines.
> > 
> > This is just for gem_sync, I forgot what did the saving for the series
> > add up to. 1-2 hours maybe?
> > 
> > > Also not sure how much curating extended.testlist is worth it, just make
> > > the testcases faster :-) Like, roughly 100x faster overall for gem_*
> > > ... >
> > > But meanwhile ack on that one too.
> > 
> > In which one, 3, or 4, or both?
> 
> Ping on the series - do we want to try easy runtime reduction via this way
> or should I drop it?

Go ahead. I'm still not happy with keeping tests around just because, but
that's a larger topic.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch