[Piglit] [RFC 00/10] Running dEQP tests without process isolation
nhaehnle at gmail.com
Wed Oct 11 10:26:49 UTC 2017
This series allows running dEQP tests (including GL CTS) without process
isolation. I've been using it quite successfully for the last few weeks,
though there are a bunch of loose ends, off the top of my head:
1. One remaining spurious failure in the `gpu` profile
2. Only tested on gpu, deqp_*, cts_gl45, khr_gl45 profiles
3. Unit tests are completely and utterly broken
4. junit backend is untested and likely needs fixes
The reason for all this is that in order to convince Piglit to run dEQP
tests without process isolation, I ended up refactoring a lot of its
internals. Most of this is documented in the commit messages, but a
1. Multiple TestResults can co-exist at the same "root" in the test
hierarchy. So you can have many TestResults that all have the root
'deqp-gles31 at functional', and subtests spread out across possibly
overlapping subgroups. This obviously involves a change in the
on-disk format for storing results.
2. Test cases are no longer grouped in the profile description. Instead,
there's the concept of a TestWorkItem, which groups a bunch of tests
that are to be run in one batch by the same TestRunner.
This means that the test case objects are no longer responsible for
running themselves; that's the domain of the separate TestRunner
hierarchy. TestRunners for shader tests and dEQP tests know how to
run multiple tests in one batch, and how to parse the test process
outputs into subtest results.
Theoretically that should also allow running shader tests from different
directories together, but I haven't done that in this series.
3. TestWorkItems are grouped deterministically in the profile run logic,
which means that resumed test runs can still have process isolation
You can find the whole series in context with some other stuff at
I probably won't be able to really polish and clean this up any time
soon, so it would be awesome if somebody would volunteer for this admittedly
unthankful task. In the meantime, the code is out there, and I hope other
people will find it useful.
More information about the Piglit