[Piglit] [PATCH 0/5] Make piglit test ordering deterministic
gabriel.feceoru at intel.com
Tue May 3 15:11:48 UTC 2016
On 03.05.2016 00:21, Dylan Baker wrote:
> This series makes the test ordering in piglit deterministic, in that it
> will run the tests in the order that they are provided by the profile.
> For native piglit tests (OpenGL and OpenCL) this will make them
> completely deterministic, for external integrated suits like dEQP and
> IGT it will be determined by the way the external suite enumerates it's
> tests; if the external tools is deterministic piglit will be too.
> An option for external suites that are not deterministic would be to
> implement a custom TestProfile class that sorts test TestDict using it's
> reorder() method.
> This further alters --test-list option to work with this internal change
> to impose ordering restrictions on the way tests are run.
> Brian, I hope this is what you're looking for.
> Daniela, I don't know exactly what you were looking for when you asked
> about sort order, but I thought I would point you at this, since Brian
> asked for something (that I think) is similar to what you're looking
> for. (https://bugs.freedesktop.org/show_bug.cgi?id=95053)
Yes, this is what we (i-g-t) were looking for.
Tested-by: Gabriel Feceoru gabriel.feceoru at intel.com
> Dylan Baker (5):
> framework/profile: Convert TestDict to a collections.MutableMapping
> framework/profile: don't replace TestDict instance with regular dict
> framework/profile: Make test run order deterministic
> framework: Make JSON backend test ordering deterministic
> framework: Make testlist option impose ordering
> framework/backends/json.py | 9 ++---
> framework/profile.py | 84 +++++++++++++++++++++++++++++++++++-----------
> framework/programs/run.py | 12 +++----
> framework/results.py | 2 +-
> unittests/profile_tests.py | 55 +++++++++++++++++++-----------
> 5 files changed, 112 insertions(+), 50 deletions(-)
More information about the Piglit