[Piglit] [RFC 00/10] Running dEQP tests without process isolation

[Marek Olšák]

This is the latest piglit branch being able to run dEQP without process
isolation. It's "rebased" on master. My definition is "rebased" is pretty
radical, but it's all I can do:

https://cgit.freedesktop.org/~mareko/piglit/log/?h=deqp

Marek

On Wed, Oct 11, 2017 at 6:26 AM, Nicolai Hähnle <nhaehnle at gmail.com> wrote:

> Hi all,
>
> This series allows running dEQP tests (including GL CTS) without process
> isolation. I've been using it quite successfully for the last few weeks,
> though there are a bunch of loose ends, off the top of my head:
>
> 1. One remaining spurious failure in the `gpu` profile
> 2. Only tested on gpu, deqp_*, cts_gl45, khr_gl45 profiles
> 3. Unit tests are completely and utterly broken
> 4. junit backend is untested and likely needs fixes
>
> The reason for all this is that in order to convince Piglit to run dEQP
> tests without process isolation, I ended up refactoring a lot of its
> internals. Most of this is documented in the commit messages, but a
> rough overview:
>
> 1. Multiple TestResults can co-exist at the same "root" in the test
>    hierarchy. So you can have many TestResults that all have the root
>    'deqp-gles31 at functional', and subtests spread out across possibly
>    overlapping subgroups. This obviously involves a change in the
>    on-disk format for storing results.
>
> 2. Test cases are no longer grouped in the profile description. Instead,
>    there's the concept of a TestWorkItem, which groups a bunch of tests
>    that are to be run in one batch by the same TestRunner.
>
>    This means that the test case objects are no longer responsible for
>    running themselves; that's the domain of the separate TestRunner
>    hierarchy. TestRunners for shader tests and dEQP tests know how to
>    run multiple tests in one batch, and how to parse the test process
>    outputs into subtest results.
>
>    Theoretically that should also allow running shader tests from different
>    directories together, but I haven't done that in this series.
>
> 3. TestWorkItems are grouped deterministically in the profile run logic,
>    which means that resumed test runs can still have process isolation
>    disabled!
>
> You can find the whole series in context with some other stuff at
> https://cgit.freedesktop.org/~nh/piglit/log/?h=deqp-run-multi
>
> I probably won't be able to really polish and clean this up any time
> soon, so it would be awesome if somebody would volunteer for this
> admittedly
> unthankful task. In the meantime, the code is out there, and I hope other
> people will find it useful.
>
> Cheers,
> Nicolai
>
> _______________________________________________
> Piglit mailing list
> Piglit at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/piglit
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/piglit/attachments/20180530/14fefb28/attachment.html>