[Piglit] [PATCH 0/5] Make piglit test ordering deterministic
dylan at pnwbakers.com
Mon May 2 21:21:09 UTC 2016
This series makes the test ordering in piglit deterministic, in that it
will run the tests in the order that they are provided by the profile.
For native piglit tests (OpenGL and OpenCL) this will make them
completely deterministic, for external integrated suits like dEQP and
IGT it will be determined by the way the external suite enumerates it's
tests; if the external tools is deterministic piglit will be too.
An option for external suites that are not deterministic would be to
implement a custom TestProfile class that sorts test TestDict using it's
This further alters --test-list option to work with this internal change
to impose ordering restrictions on the way tests are run.
Brian, I hope this is what you're looking for.
Daniela, I don't know exactly what you were looking for when you asked
about sort order, but I thought I would point you at this, since Brian
asked for something (that I think) is similar to what you're looking
Dylan Baker (5):
framework/profile: Convert TestDict to a collections.MutableMapping
framework/profile: don't replace TestDict instance with regular dict
framework/profile: Make test run order deterministic
framework: Make JSON backend test ordering deterministic
framework: Make testlist option impose ordering
framework/backends/json.py | 9 ++---
framework/profile.py | 84 +++++++++++++++++++++++++++++++++++-----------
framework/programs/run.py | 12 +++----
framework/results.py | 2 +-
unittests/profile_tests.py | 55 +++++++++++++++++++-----------
5 files changed, 112 insertions(+), 50 deletions(-)
More information about the Piglit