[Piglit] Nearly finished: shader_runner running THOUSANDS of tests per process

[Dylan Baker]

Quoting Marek Olšák (2016-04-16 15:16:34)
> Hi,
> 
> This makes shader_runner very fast. The expected result is 40%
> decrease in quick.py running time, or a 12x faster piglit run if you
> run shader tests alone.
> 
> Branch:
> https://cgit.freedesktop.org/~mareko/piglit/log/?h=shader-runner
> 
> Changes:
> 
> 1) Any number of test files can be specified as command-line
> parameters. Those command lines can be insanely long.
> 
> 2) shader_runner can re-create the window & GL context if test
> requirements demand different settings when going from one test to
> another.
> 
> 3) all.py generates one shader_runner instance per group of tests
> (usually one or two directories - tests and generated_tests).
> Individual tests are reported as subtests.
> 
> The shader_runner part is done. The python part needs more work.
> 
> 
> What's missing:
> 
> Handling of crashes. If shader_runner crashes:
> - The crash is not shown in piglit results (other tests with subtests
> already have the same behavior)
> - The remaining tests will not be run.
> 
> The ShaderTest python class has the list of all files and should be
> able to catch a crash, check how many test results have been written,
> and restart shader_runner with the remaining tests.
> 
> shader_runner prints TEST %i: and then the subtest result. %i is the
> i-th file in the list. Python can parse that and re-run shader_runner
> with the first %i tests removed. (0..%i-1 -> parse subtest results; %i
> -> crash; %i+1.. -> run again)
> 
> 
> I'm by no means a python expert, so here's an alternative solution (for me):
> - Catch crash signals in shader_runner.
> - In the single handler, re-run shader_runner with the remaining tests.
> 
> Opinions welcome,
> 
> Marek
> _______________________________________________
> Piglit mailing list
> Piglit at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/piglit

Hey Marek,

I'd picked this up and was finishing it up, I have a branch on my github
(https://github.com/dcbaker/piglit wip/multi-shader_runner), I'm just
trying to make sure we're not duplicating effort.

Dylan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: signature
URL: <https://lists.freedesktop.org/archives/piglit/attachments/20160527/93176ce3/attachment-0001.sig>