[Piglit] [PATCH] tests/quick.py: reduce the number of vs_in tests in quick.py

[Andres Gomez]

On Wed, 2016-11-09 at 14:56 -0800, Dylan Baker wrote:
> Bump.
> 
> Anyone interested in this?
> 
> 

As commented, although this is not a definitive solution I suppose this
can help in the meanwhile.

I'm sorry I didn't contribute a follow up. I got tangled in other tasks
and never found time to get to it.

In any case, my idea was to reduce the amount of tested cases since I
think right now there are probably many tests that check very similar
inputs. Additionally, I was thinking on something like this for the
quick profile so this works for me.

However, maybe a C test could be a better idea, dunno.

>From my side, this is:

Reviewed-by: Andres Gomez <agomez at igalia.com>

> 
> Dylan
> 
> Quoting Dylan Baker (2016-10-31 14:40:35)
> > There are 18000 of these tests, quick.py has 53000 tests, which is 34%
> > of the total tests. That's just too many for quick.py.
> > 
> > This patch takes the somewhat naive approach of just filtering out 80%
> > of those tests from quick.py, which reduces the total number of tests to
> > 38000, which makes a pretty big dent in the amount of time it takes to
> > run quick.py. The approach is deterministic, but random.
> > 
> > cc: Andres Gomez <agomez at igalia.com>
> > Signed-off-by: <dylanx.c.baker at intel.com>
> > ---
> >  tests/quick.py | 27 +++++++++++++++++++++++++++
> >  1 file changed, 27 insertions(+)
> > 
> > diff --git a/tests/quick.py b/tests/quick.py
> > index 0e02f92..5f7250f 100644
> > --- a/tests/quick.py
> > +++ b/tests/quick.py
> > @@ -1,8 +1,19 @@
> >  # -*- coding: utf-8 -*-
> >  
> > +"""A quicker profile than all.
> > +
> > +This profile filters out a number of very slow tests, and tests that are very
> > +exhaustively tested, since they add a good deal of runtime to piglit.
> > +
> > +There are 18000+ auto-generated tests that are exhaustive, but for quick.py we
> > +don't want that level of exhaustiveness, so this filter removes 80% in a random
> > +(but deterministic) way.
> > +"""
> > +
> >  from __future__ import (
> >      absolute_import, division, print_function, unicode_literals
> >  )
> > +import random
> >  
> >  from framework import grouptools
> >  from framework.test import (GleanTest, PiglitGLTest)
> > @@ -13,6 +24,21 @@ __all__ = ['profile']
> >  # See the note in all.py about this warning
> >  # pylint: disable=bad-continuation
> >  
> > +
> > +class FilterVsIn(object):
> > +    """Filter out 80% of the Vertex Attrib 64 vs_in tests."""
> > +
> > +    def __init__(self):
> > +        self.random = random.Random()
> > +        self.random.seed(42)
> > +
> > +    def __call__(self, name, _):
> > +        if 'vs_in' in name:
> > +            # 20%
> > +            return self.random.random() <= .2
> > +        return True
> > +
> > +
> >  GleanTest.GLOBAL_PARAMS += ["--quick"]
> >  
> >  # Set the --quick flag on a few image_load_store_tests
> > @@ -37,3 +63,4 @@ with profile.group_manager(
> >  
> >  # These take too long
> >  profile.filter_tests(lambda n, _: '-explosion' not in n)
> > +profile.filter_tests(FilterVsIn())
> > -- 
> > 2.10.2
> > 
-- 
Br,

Andres