[Piglit] [RFC v2 00/39] Rework profile modification (all.py)

[Dylan Baker]

This is the second go around of this series, I'm still hoping to get
some feedback as to whether the community likes the approach.

This version incorporates changes suggested by Ilia, require_platforms
and exclude_platforms are now easier to work with. This version also
includes patches to flip the meaning of run_concurrent for PiglitGLTest,
ShaderTest, and GLSLParserTest. It also simplifies the way that
PiglitCLTest handles it's run_concurrent attribute.

One suggestion from Ilia that I implemented but ultimately decided
against was the ability to specify a prefix to be removed or added to
the arguments. To me it's very important that the arguments to the test
be exact and explicit, there should not be any magic going on, since
these are what makes the test, so specifying a prefix to remove from the
arguments to make the name of the test seemed fine, however, the number
of tests that actually would make use of the feature was relatively
small, and it required in many cases calling group_manager multiple
times for a single group. It also has a good chance of surprising
someone, so I opted to not do that.

Dylan Baker (39):
  test_lists.py: remove tests for r300, r500, and all_cl which are gone
  test_lists.py: add cput and llvmpipe profiles.
  tests/utils.py: Add a Test derived class for unit tests
  all.py: fix misnamed tests
  delete tests/es3conform.py
  profiles: add new profiles that only run specific test classes.
  profile.py: Use a Tree structure for Profile.tests
  profile.py: Lower all test keys before getting and setting.
  shader_test.py: Removed recursive keyword from add_shader_test_dir
  shader_test.py: remove add_shader_test
  shader_test.py: use os.path.splitext instead of rsplit('.')
  shader_test.py: use os.walk in add_shader_test_Dir
  shader_test.py: convert add_shader_test_dir to use a flat group
  all.py: Replace use of add_glsl_parser_test with
    import_glsl_parser_test
  glsl_parser_test.py: remove add_glsl_parser_test
  all.py: replace add_shader_test_dir with loop in all.py
  framework: update json version to v4 to fix test changes
  all.py: replace import_glsl_parser_test with loop search
  all.py: remove extra newline in list.
  all.py: use profile.test_list instead of profile.test for glean tests
  all.py: move all of the helper functions to the top of the file
  profile.py: Add a context_manager to TestProfile for adding tests.
  all.py: Convert all.py to use TestProfile.group_manager
  all.py: add asmparser tests automatically with loop
  sanity.py: Use TestProfile.group_manager
  cl.py: Use TestProfile.group_manager
  cl.py: simplify add_program_test_dir
  cl.py: Use flat list for program test dirs
  opencv.py: use test_list instead of tests.
  profile.py: Remove TestProfile.tests
  profile.py: Add kwargs to TestProfile.group_manager
  all.py: Use TestProfile.group_manager default_args to set
    require_platforms
  all.py: Use group_manager default_args for exclude_platforms
  framework: Override run_concurrent for CL test classes
  all.py: Set run_concurrent=False explicitly
  piglit_test.py: Change run_concurrent to True by default
  all.py: Remove run_concurrent=True flag
  all.py: use __future__ functions
  all.py: disable pylint's bad-continuation error

 README                                    |    6 +
 framework/backends/json.py                |    2 +-
 framework/profile.py                      |  132 +-
 framework/results.py                      |   55 +
 framework/test/glsl_parser_test.py        |   69 +-
 framework/test/opencv.py                  |    7 +-
 framework/test/piglit_test.py             |   26 +-
 framework/test/shader_test.py             |   34 -
 framework/tests/glsl_parser_test_tests.py |    4 +-
 framework/tests/profile_tests.py          |  161 +-
 framework/tests/results_v2_tests.py       |   92 +
 framework/tests/results_v3_tests.py       |  134 +
 framework/tests/shader_test_tests.py      |   11 -
 framework/tests/test_lists.py             |    4 +-
 framework/tests/utils.py                  |   80 +
 tests/all.py                              | 7905 ++++++++++++++---------------
 tests/cl.py                               |  205 +-
 tests/es3conform.py                       |   84 -
 tests/glslparser.py                       |    8 +
 tests/gpu.py                              |    6 +-
 tests/llvmpipe.py                         |   21 +-
 tests/quick.py                            |    5 +-
 tests/sanity.py                           |    6 +-
 tests/shader.py                           |    8 +
 24 files changed, 4520 insertions(+), 4545 deletions(-)
 create mode 100644 framework/tests/results_v2_tests.py
 create mode 100644 framework/tests/results_v3_tests.py
 delete mode 100644 tests/es3conform.py
 create mode 100644 tests/glslparser.py
 create mode 100644 tests/shader.py

-- 
2.2.2