[Piglit] [PATCH v4 0/27] Various cleanups for profiles

[Dylan Baker]

This series is a large collections of cleanups for th profile module and
particularly for the TestProfile and TestDict classes. These began their
lives as the start of a series to serialize the TestDict at compile time
and to move startup overhead.

This series is just a bunch of cleanups and removes a *lot* of code, and
simplifies a number of things, as well as fixing a few long standing
bugs in piglit. The biggest of these fixes is that it is now possible to
pass two derivatives of all.py (say glslparser.py and shader.py) at the
same time.

This series passes our CI system with no changes or regressions.

This has gone out several times with no comments, I'm planning to push first
thing in the morning if no one objects.

Changes in v2:
    - Rebase on master, including one patch that was part of this
      series, but was merged since it fixed a bug
    - Don't add filters that do nothing in framework/programs/run.py.
      This doesn't really affect this series, but it does have
      ramifications in some other work that isn't ready to go to the
      list yet.

Changes in v3:
    - Don't close the pools until all of the profiles have finished
      running
    - Don't remove the _{pre,post}_run_hook methods, instead rename them
      to setup and teardown, making them public. They actually are used
      by XTS and IGT, which I didn't notice. I still think I'd like to
      get rid of them, but not while they're in use and there isn't an
      obviously good way to do so.
    - Store images in the JSON file, rather than externally
    - the XTS profile uses a temporary directory to store it's results,
      which (along with the previous change) removes the need to pass
      the results directory to its tests.
    - Fix RegexFilter to work for both the include filter (-t) and the
      exclude filter (-x), it previously only worked for one or the
      other. Also add tests.

Changes in v4:
    - Add tests for summary-feature
    - Don't break summary feature tests with changes (previously this
      was component was completely broken)
    - Fix the inverse flag in RegexFilter (in framework/program/run.py),
      which was being passed to the wrong function.
    - Remove RFC patch to use enum34 (and fix changes resulting from that
      removal)
    - Fix args.concurrent, which should be args.concurrency, in programs/run.py
    - Fix check for monitoring errors in profile.run

Dylan Baker (27):
  framework/result: TestrunResult.options should be a dict
  unittests: Add tests for the feature completeness summary
  framework/profile: make run hooks public
  framework/profile: Drop unneeded except
  framework: Split the run method out of profile.
  framework: remove concurrent from OPTIONS, pass directly to profile
  framework/programs/run: Remove use of TestrunResult
  framework/profile: add a copy method to profile
  tests: Copy profiles to allow them to be run in parallel
  framework/programs/run: Only allow --test-list if one profile
  framework/profile: Don't merge profiles
  framework/programs/run: remove redundant list() around comprehension
  framework/profile: Drop TestProfile.filter_tests
  framework/programs/run: import framework.profile as profile
  framework: Remove exclude_tests from options.OPTIONS
  framework: Pull {include,exclude}_filter out of Options
  framework/profile: Factor out check_all closure
  framework/profile: Don't alter the TestProfile before running
  framework/profile: Update __all__
  framework/profile: Update docstrings
  framework/profile: Split try/except block
  framework/profile: replace Testprofile.{dmesg,monitoring} with dict
  framework/profile: remove unused argument from TestProfile.group_manager
  framework/profile: Move group_manager from TestProfile to TestDict
  framework/test/base: Remove timeout parameter
  framework/test/glsl_parser_test.py: split the parser out of the class
  tests/all.py: Make add_fbo_depthstencil_tests take an adder

 framework/options.py                         | 144 +-----
 framework/profile.py                         | 564 ++++++++-----------
 framework/programs/print_commands.py         |  13 +-
 framework/programs/run.py                    | 113 ++--
 framework/results.py                         |   2 +-
 framework/summary/feature.py                 |  30 +-
 framework/test/base.py                       |  36 +-
 framework/test/glsl_parser_test.py           |  85 +--
 tests/all.py                                 | 605 +++++++++-----------
 tests/cl.py                                  |   8 +-
 tests/cpu.py                                 |   6 +-
 tests/glslparser.py                          |   6 +-
 tests/gpu.py                                 |   8 +-
 tests/igt.py                                 |  10 +-
 tests/llvmpipe.py                            |   4 +-
 tests/quick.py                               |  14 +-
 tests/quick_cl.py                            |   4 +-
 tests/sanity.py                              |   2 +-
 tests/shader.py                              |   6 +-
 tests/xts-render.py                          |   9 +-
 tests/xts.py                                 |   8 +-
 unittests/framework/summary/test_feature.py  | 115 ++++-
 unittests/framework/test/test_base.py        |  14 +-
 unittests/framework/test/test_shader_test.py |   2 +-
 unittests/framework/test_options.py          | 178 +------
 unittests/framework/test_profile.py          | 363 ++++--------
 26 files changed, 1008 insertions(+), 1341 deletions(-)
 create mode 100644 unittests/framework/summary/test_feature.py

-- 
git-series 0.8.10