[Piglit] [PATCH 0/5] Make piglit test ordering deterministic

[Dylan Baker]

This series makes the test ordering in piglit deterministic, in that it
will run the tests in the order that they are provided by the profile.
For native piglit tests (OpenGL and OpenCL) this will make them
completely deterministic, for external integrated suits like dEQP and
IGT it will be determined by the way the external suite enumerates it's
tests; if the external tools is deterministic piglit will be too.

An option for external suites that are not deterministic would be to
implement a custom TestProfile class that sorts test TestDict using it's
reorder() method.

This further alters --test-list option to work with this internal change
to impose ordering restrictions on the way tests are run.

Brian, I hope this is what you're looking for.

Daniela, I don't know exactly what you were looking for when you asked
about sort order, but I thought I would point you at this, since Brian
asked for something (that I think) is similar to what you're looking
for. (https://bugs.freedesktop.org/show_bug.cgi?id=95053)

Dylan Baker (5):
  framework/profile: Convert TestDict to a collections.MutableMapping
  framework/profile: don't replace TestDict instance with regular dict
  framework/profile: Make test run order deterministic
  framework: Make JSON backend test ordering deterministic
  framework: Make testlist option impose ordering

 framework/backends/json.py |  9 ++---
 framework/profile.py       | 84 +++++++++++++++++++++++++++++++++++-----------
 framework/programs/run.py  | 12 +++----
 framework/results.py       |  2 +-
 unittests/profile_tests.py | 55 +++++++++++++++++++-----------
 5 files changed, 112 insertions(+), 50 deletions(-)

-- 
2.8.2