[Piglit] Rework profiles

Dylan Baker baker.dylan.c at gmail.com
Tue Jan 27 14:58:34 PST 2015

This is a very large, very invasive rework of the way that test profiles
work, hopefully for the better. The goal was to create a simple syntax
that discouraged variation, and limited accidents. What I ended up with
is a two fold change in the way tests are added.

First, shader tests, glslparser tests, and asmparser tests are
automatically added to the profile with *zero* modification to all.py (A
similar setup could be used by cl.py, but I wanted the input of the cl
devs, since it would mean either renaming or moving tests). These tests
use their directory structure to determine what groups they belong to.

This necessitated another change, treating all group and test names as
case insensitive. This will also fix the problem on case sensitive file
systems of having tests split across groups in the html summary.

The second change it the removal of add_plain_test and
add_concurrent_test, they are replaced by a method in TestProfile that
gives an 'adder' function, which encompasses all the power of
add_concurrent_test and add_plain_test, but also has all of the power of
adding tests directly to the profile.

The third change is the removal of TestProfile.tests, the nested
dictionary structure which was flattened before running, and replacing
it with the use of TestProfile.test_list, a dictionary of string:Test
values that assumes that the tests are already formatted appropriately,
this series of changes makes extensive use of the grouptools module to
ensure future flexibility in this regard.

Patches 1-2 are out as a separate series.
Patches 3-5 are groundwork changes
Patch 6 adds two new profiles that I have carried around for some time
and are very useful for development
Patches 7-30 Are the bulk of the series.

Patches 16-18 will need to be merged into one mega-patch to allow clean
bisect. I have tried to come up with a better way to do this, but can't
come up with any other solution that allows bisecting all portions of
piglit except the mega patch

The end result, even with additional testing and two new results
versions (which add a lot of code for updating and validating), is ~+500
LOC, even with my efforts to wrap lines better in all.py then they were

This is available at my github:
https://github.com/dcbaker/piglit rfc/flat-test-list

More information about the Piglit mailing list