[Piglit] [PATCH 00/11] OpenGL fast skipping for native tests.

[baker.dylan.c at gmail.com]

From: Dylan Baker <baker.dylan.c at gmail.com>

One of the problems with piglit is the runtime, especially on older
platforms where large numbers of tests skip. The average runtime on HSW
for a shader_runner test that skips is about .05 seconds. This seems
pretty small, until one considers that there are ~3000 skipping shader
tests, and that number will only grow as more feature are added to
piglit.

This patch implements a Mixin class for Test derived classes that
implements skipping based on OpenGL and OpenGL ES extensions, version,
and shader language version. It relies on wflinfo for this information,
but will fall back if wflinfo isn't available.

This mixin is provided to ShaderTest and GLSLParserTest, which already
have this information readily available to them (or can easily gather
it), and improves their run time on all platforms substantially.

On g965, this reduced piglit runtime for gpu.py by roughly 25%, using
'-c' with two cores.

Patches 1-2: My global options series which has been on the list,
             I'll likely merge that soon unless there are objections
Patch 3:     Fixes a bug in one of the test generators

The remaining patches implement the fast skipping feature.

The design of this mixin is conservative. I would prefer to run a few
tests that could be skipped, but never skip one that could be run than
the opposite.

It might also be worth having (though it is no way required) my patch to
update glslparser to know about GLES 3.1 and GLES 3.2, since without it
one test (at least for me) will go from fail to skip.

Dylan Baker (11):
  framework: make options a global variable.
  framework: Convert the codebase to use the new global Options
  gen_builtin_uniform_tests.py: Add extensions to require section
  framework/test/opengl.py: Add FastSkipMixin which checks extensions
  framework/test/opengl.py: add support for GL(ES) version skipping.
  framework/test/opengl.py: Add GLSL and GLSL ES fast skipping
  framework/test/shader_test.py: Add fast skipping support for
    extensions
  framework/test/shader_test.py: add GL and GLES version fast skipping.
  framework/test/shader_test.py: add GLSL (ES) based skipping.
  framework/test/glsl_parser_test.py: Add requirement based fast
    skipping
  framework/test/glsl_parser_test.py: add support for GLSL (ES) based
    skipping.

 framework/backends/abstract.py               |  16 +-
 framework/core.py                            |  50 ----
 framework/options.py                         | 185 +++++++++++++
 framework/profile.py                         |  55 ++--
 framework/programs/run.py                    |  62 ++---
 framework/test/base.py                       |  14 +-
 framework/test/gleantest.py                  |   6 +-
 framework/test/glsl_parser_test.py           |  20 +-
 framework/test/opengl.py                     | 338 +++++++++++++++++++++++
 framework/test/piglit_test.py                |   4 +-
 framework/test/shader_test.py                | 104 +++++--
 framework/tests/backends_tests.py            |   8 +-
 framework/tests/base_tests.py                |  43 ++-
 framework/tests/core_tests.py                |  27 +-
 framework/tests/gleantest_tests.py           |  17 +-
 framework/tests/glsl_parser_test_tests.py    |  75 ++++-
 framework/tests/json_tests.py                |   8 +-
 framework/tests/opengl_tests.py              | 399 +++++++++++++++++++++++++++
 framework/tests/options_tests.py             | 178 ++++++++++++
 framework/tests/piglit_test_tests.py         |  26 +-
 framework/tests/profile_tests.py             |  33 ++-
 framework/tests/shader_test_tests.py         | 117 ++++++++
 generated_tests/gen_builtin_uniform_tests.py |   2 +
 piglit-print-commands.py                     |  12 +-
 tests/igt.py                                 |   6 +-
 tests/xts.py                                 |   2 +-
 tox.ini                                      |   4 +-
 27 files changed, 1555 insertions(+), 256 deletions(-)
 create mode 100644 framework/options.py
 create mode 100644 framework/test/opengl.py
 create mode 100644 framework/tests/opengl_tests.py
 create mode 100644 framework/tests/options_tests.py

-- 
2.6.2