[Piglit] [PATCH 2/2] all.py: filter directories traversed to find shader tests

[Brian Paul]

On 11/09/2017 05:29 PM, Dylan Baker wrote:
> Quoting Brian Paul (2017-11-09 15:16:33)
>> The script searches all files under tests/ and generated_tests/ for
>> files ending in ".shader_test".  For each match, a ShaderTest() object
>> is created and added to the test list.
>>
>> For GL extensions or versions not supported by the driver, this is
>> wasted effort.
>>
>> This patch looks for directories under spec/ and tries to determine if
>> the extension/feature is actually supported by the current driver.  If
>> not, it's skipped.
>>
>> This somewhat reduces Piglit start up time, but substantially more
>> time is spent in the ShaderTest() constructor which actually opens
>> and parses each file to determine its GL/ES dependencies.  I'll try
>> to address that in the future.
>>
>> v2:
>> - replace os.path.walk() with my own walk_dir_tree() which avoids
>>    decending into subdirs when the parent directory/feature is not supported.
>> - Use env var to enable/disable shader directory filtering
>> - Also, fix naming conventions and minor formatting issues.
>> ---
>>   tests/all.py | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>>   1 file changed, 72 insertions(+), 1 deletion(-)
>>
>> diff --git a/tests/all.py b/tests/all.py
>> index ae4a995..93f66cf 100644
>> --- a/tests/all.py
>> +++ b/tests/all.py
>> @@ -8,12 +8,15 @@ import collections
>>   import itertools
>>   import os
>>   import platform
>> +import re
>>
>>   import six
>>   from six.moves import range
>>
>>   from framework import grouptools
>> +from framework.test import opengl
>>   from framework import options
>> +from framework import wflinfo
>>   from framework.profile import TestProfile
>>   from framework.driver_classifier import DriverClassifier
>>   from framework.test import (PiglitGLTest, GleanTest, PiglitBaseTest,
>> @@ -210,9 +213,77 @@ profile = TestProfile()  # pylint: disable=invalid-name
>>
>>   shader_tests = collections.defaultdict(list)
>>
>> +wfl_info = wflinfo.WflInfo()
>> +
>> +
>> +def gl_extension_supported(ext_name):
>> +    """Is the named OpenGL extension supported?"""
>> +    return ext_name in wfl_info.gl_extensions
>
> There should be one extra newline here.
>
>> +
>> +def is_feature_directory_supported(dir_name):
>> +    """Determine if dir_name specifies an OpenGL feature (extension or GL
>> +    version) which is supported by the host.  If we return False, it means
>> +    the extension/version is definitely not supported.  If we return True,
>> +    it means the extension/version is possibly suppported.  We're a little
>> +    fuzzy because we don't yet parse all the directory name possibilities
>> +    (like ES tests).
>> +    """
>> +    if dir_name[:4] in {"amd_", "arb_", "ati_", "ext_", "khr_", "oes_"}:
>> +        # The directory is a GL extension name, but of the format "arb_foo_bar"
>> +        # instead of "GL_ARB_foo_bar".  We convert the former into the later
>> +        # and check if the extension is supported.
>> +        ext_name = "GL_" + dir_name[0:4].upper() + dir_name[4:]
>> +        return gl_extension_supported(ext_name)
>> +    elif re.match("gl-\d\.\d+", dir_name):
>
> You're using `re` here because you're not checking gl[sl]-es, right? Does it
> make more sense to handle ES before desktop GL so you don't need to use re?

Yeah, I can flesh out the ES cases and get rid of re.


>
>> +        # The directory is a GL version
>> +        version = dir_name[3:]
>> +        return float(version) <= float(wfl_info.gl_version)
>> +    elif re.match("glsl-\d\.\d+", dir_name):
>> +        # The directory is a GLSL version
>> +        version = dir_name[5:]
>> +        return float(version) <= float(wfl_info.glsl_version)
>> +    else:
>> +        # The directory is something else.  Don't skip it.
>> +        return True
>> +
>> +
>> +def walk_filter_dir_tree(root):
>> +    """Recursively walk the directory tree rooted at 'root'.
>> +    If we find a directory path of the form ".../spec/foo/" we'll check if
>> +    'foo' is a supported extension/feature/version.  If not, we do not
>> +    traverse foo/.  Otherwise, we add continue traversing.
>> +    The return value is a list of (dirpath, filename) tuples.
>> +    """
>> +    curdir = os.path.split(root)[1]
>> +    files = []
>> +    retval = []
>> +
>> +    for entry in os.listdir(root):
>> +        full_path = os.path.join(root, entry)
>> +        if os.path.isdir(full_path):
>> +            # Check if we're in a "spec/" direcotry
>> +            if curdir == "spec" and not is_feature_directory_supported(entry):
>> +                # The directory's tests aren't supported by the driver.
>> +                print("Skipping spec/{}".format(entry))
>> +            else:
>> +                # recursively walk the subdirectory
>> +                retval += walk_filter_dir_tree(full_path)
>> +        elif os.path.isfile(full_path):
>> +            # Add the file to the files list
>> +            files += [entry]
>> +
>> +    retval += [(root, None, files)]
>
> It wont affect anything in our use, but to address potential future problems,
> None should be [] here.
>
>> +
>> +    return retval
>
> Just a comment, os.walk is a generator, it yields values instead of returning
> a sequence. The way you're using them they're interchangeable, but they are not
> universally interchangeable. It also means that this will much more memory
> intensive than os.walk.

Yes, I'm aware of that, but it's not all that large.  For "test/" it's a 
list of 406 elements and for "generated_tests/" it's 163.  And the 
number of filenames in each element ranges from 1 to 1487 with the 
average being pretty small.

v3 coming.

-Brian

>
>> +
>> +
>>   # Find and add all shader tests.
>>   for basedir in [TESTS_DIR, GENERATED_TESTS_DIR]:
>> -    for dirpath, _, filenames in os.walk(basedir):
>> +    if os.environ.get("PIGLIT_FILTER_DIRECTORIES"):
>> +        files = walk_filter_dir_tree(basedir)
>> +    else:
>> +        files = os.walk(basedir)
>> +    for dirpath, _, filenames in files:
>>           groupname = grouptools.from_path(os.path.relpath(dirpath, basedir))
>>           for filename in filenames:
>>               testname, ext = os.path.splitext(filename)
>> --
>> 1.9.1
>>