[Piglit] [PATCH 0/5] Make piglit test ordering deterministic

[Gabriel Feceoru]

On 03.05.2016 00:21, Dylan Baker wrote:
> This series makes the test ordering in piglit deterministic, in that it
> will run the tests in the order that they are provided by the profile.
> For native piglit tests (OpenGL and OpenCL) this will make them
> completely deterministic, for external integrated suits like dEQP and
> IGT it will be determined by the way the external suite enumerates it's
> tests; if the external tools is deterministic piglit will be too.
>
> An option for external suites that are not deterministic would be to
> implement a custom TestProfile class that sorts test TestDict using it's
> reorder() method.
>
> This further alters --test-list option to work with this internal change
> to impose ordering restrictions on the way tests are run.
>
> Brian, I hope this is what you're looking for.
>
> Daniela, I don't know exactly what you were looking for when you asked
> about sort order, but I thought I would point you at this, since Brian
> asked for something (that I think) is similar to what you're looking
> for. (https://bugs.freedesktop.org/show_bug.cgi?id=95053)

Yes, this is what we (i-g-t) were looking for.
Thank you!

Tested-by: Gabriel Feceoru gabriel.feceoru at intel.com

>
> Dylan Baker (5):
>    framework/profile: Convert TestDict to a collections.MutableMapping
>    framework/profile: don't replace TestDict instance with regular dict
>    framework/profile: Make test run order deterministic
>    framework: Make JSON backend test ordering deterministic
>    framework: Make testlist option impose ordering
>
>   framework/backends/json.py |  9 ++---
>   framework/profile.py       | 84 +++++++++++++++++++++++++++++++++++-----------
>   framework/programs/run.py  | 12 +++----
>   framework/results.py       |  2 +-
>   unittests/profile_tests.py | 55 +++++++++++++++++++-----------
>   5 files changed, 112 insertions(+), 50 deletions(-)
>