[Piglit] [PATCH 0/5] Make piglit test ordering deterministic
brianp at vmware.com
Mon May 2 23:24:38 UTC 2016
On 05/02/2016 03:21 PM, Dylan Baker wrote:
> This series makes the test ordering in piglit deterministic, in that it
> will run the tests in the order that they are provided by the profile.
> For native piglit tests (OpenGL and OpenCL) this will make them
> completely deterministic, for external integrated suits like dEQP and
> IGT it will be determined by the way the external suite enumerates it's
> tests; if the external tools is deterministic piglit will be too.
> An option for external suites that are not deterministic would be to
> implement a custom TestProfile class that sorts test TestDict using it's
> reorder() method.
> This further alters --test-list option to work with this internal change
> to impose ordering restrictions on the way tests are run.
> Brian, I hope this is what you're looking for.
Yes, it is. Thanks!
I applied the patchs and tried a few test runs and it looks good.
I just did a light review of the code changes.
Acked-by: Brian Paul <brianp at vmware.com>
Tested-by: Brian Paul <brianp at vmware.com>
One other issue though. It would be nice if the output of
piglit-print-commands.py could be used as a test list input to
piglit-run.py When I tried it, piglit-run.py spun its wheels for a
while then hit an exception.
> Daniela, I don't know exactly what you were looking for when you asked
> about sort order, but I thought I would point you at this, since Brian
> asked for something (that I think) is similar to what you're looking
> for. (https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.freedesktop.org_show-5Fbug.cgi-3Fid-3D95053&d=BQIBAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=T0t4QG7chq2ZwJo6wilkFznRSFy-8uDKartPGbomVj8&m=P0idU-rSN_1ROsAgtRw0-jqXAQvrw1lv886Sa7DT818&s=KhSdC9bxSuoRx9psA_FmzE8UIKCcTtyWCzT7zBXOQ08&e= )
> Dylan Baker (5):
> framework/profile: Convert TestDict to a collections.MutableMapping
> framework/profile: don't replace TestDict instance with regular dict
> framework/profile: Make test run order deterministic
> framework: Make JSON backend test ordering deterministic
> framework: Make testlist option impose ordering
> framework/backends/json.py | 9 ++---
> framework/profile.py | 84 +++++++++++++++++++++++++++++++++++-----------
> framework/programs/run.py | 12 +++----
> framework/results.py | 2 +-
> unittests/profile_tests.py | 55 +++++++++++++++++++-----------
> 5 files changed, 112 insertions(+), 50 deletions(-)
More information about the Piglit