[Piglit] [PATCH 0/5] Make piglit test ordering deterministic

[Marius Vlad]

So, with these patches we won't have that ``randomized'' feature at all
right? (unless the test list is given as such).

On Mon, May 02, 2016 at 02:21:09PM -0700, Dylan Baker wrote:
> This series makes the test ordering in piglit deterministic, in that it
> will run the tests in the order that they are provided by the profile.
> For native piglit tests (OpenGL and OpenCL) this will make them
> completely deterministic, for external integrated suits like dEQP and
> IGT it will be determined by the way the external suite enumerates it's
> tests; if the external tools is deterministic piglit will be too.
> 
> An option for external suites that are not deterministic would be to
> implement a custom TestProfile class that sorts test TestDict using it's
> reorder() method.
> 
> This further alters --test-list option to work with this internal change
> to impose ordering restrictions on the way tests are run.
> 
> Brian, I hope this is what you're looking for.
> 
> Daniela, I don't know exactly what you were looking for when you asked
> about sort order, but I thought I would point you at this, since Brian
> asked for something (that I think) is similar to what you're looking
> for. (https://bugs.freedesktop.org/show_bug.cgi?id=95053)
> 
> Dylan Baker (5):
>   framework/profile: Convert TestDict to a collections.MutableMapping
>   framework/profile: don't replace TestDict instance with regular dict
>   framework/profile: Make test run order deterministic
>   framework: Make JSON backend test ordering deterministic
>   framework: Make testlist option impose ordering
> 
>  framework/backends/json.py |  9 ++---
>  framework/profile.py       | 84 +++++++++++++++++++++++++++++++++++-----------
>  framework/programs/run.py  | 12 +++----
>  framework/results.py       |  2 +-
>  unittests/profile_tests.py | 55 +++++++++++++++++++-----------
>  5 files changed, 112 insertions(+), 50 deletions(-)
> 
> -- 
> 2.8.2
> 
> _______________________________________________
> Piglit mailing list
> Piglit at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/piglit
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: Digital signature
URL: <https://lists.freedesktop.org/archives/piglit/attachments/20160503/ab2fa1fc/attachment.sig>