[Piglit] deqp: speed up piglit/dEQP with a group-at-a-time mode

Dylan Baker baker.dylan.c at gmail.com
Thu Mar 24 18:43:00 UTC 2016

In our CI system we use piglit to tid dEQP and jenkins together for a
number of reasons. Some of it is getting JUnit reporting, and some of it
is the reliablity of having processes isolation. However, performance is
becoming a problem for us.

Looking at piglit/dEQP runner the biggest overhead is in process startup
overhead. Google does not run dEQP in test-at-a-time configurations like
piglit does, they haven't optimized it. There's a lot of cost there.

In order to reduce this cost I've created a new mode, which I call
'group-at-a-time' or just 'group'. This new mode for dEQP is added in
addition to the current 'test-at-a-time' mode, and is enabled with
command line switches. When run in this mode the dEQP tests are not
processed looking for tests, but for the group leaves.

For example, given the following structure:
    GROUP: dEQP-GLES2.info
    TEST: dEQP-GLES2.info.vendor
    TEST: dEQP-GLES2.info.version

test-at-a-time mode would create two new test instances, one for
'dEQP-GLES2.info.vendor', and one for 'dEQP-GLES2.info.version'. The
group-at-a-time mode would create a single test for 'dEQP-GLES2.info.*'

This group at a time mode uses piglit's subtest reporting to report each
test result, which allows the normal piglit comparison tools to be used
to compare test-at-a-time results and group-at-a-time results

More information about the Piglit mailing list