[Piglit] Proposal to embed glslparsertest specifications into shader source files

[Chad Versace]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

(A proposal to make Piglit a little easier to use... Please send me your
feedback.)

The format for glslparsertests is suboptimal; the test specification
(pass/fail, required GLSL version) is not located in the test file
itself. This divorce between test and specification causes frequent
maintenance and testing annoyances (How many times have you opened a
failing glslparsertest shader file, only to discover that the author
neglected to comment whether the test should pass or fail?)

I propose that we embed each test specification into the shader source
file itself as a specially formatted comment block, and cease to
register the specifications in /tests/glslparsertest.tests.

What follows is an overview of the new test file format and of the
proposal's implementation.

As for the work involved, it has already been done. In an experimental
Piglit branch [1] [2], all of the below has been implemented and all
pre-existing glslparertests converted to the new format. No testing
quirks have been introduced; test results are identical between master
and the experimental branch.


=== What's in it for me ;) ===

I will soon be writing many many GLSL 1.30 parser tests, and would like
that task to as sane as possible.


=== Comment Block Format ===

The special comment block is simply a list of option/value pairs that
begins with the line '[config]'.

    /* Test bit-shift on ints.
     *
     * [config]
     * expect_result: pass
     * glsl_version: 130
     * [end config]
     *
     * author: Joe Programmer
     */

    int f() {
        return 1 << 4;
    }

The format is that of ConfigParser.SafeConfigParser in the Python
stdlib. By leveraging a pre-existing class in the stdlib:
    * We avoid the messy task of implementing an adhoc parsing scheme.
    * Adding future support for additional option/value pairs is
      trivial.


=== Test Driver ===

The test driver is a new class GLSLParserTest in
/framework/glsl_parser_test.py. It performs just two tasks:
    1. Extract the test specification from the shader source file
       (and possibly report errors in the special comment block).
    2. Pass the source file and specifications to /bin/glslparsertest
       (which is exactly what Piglit does now).

As a bonus feature, glsl_parser_test.py is executable and
hence can be ran as a stand-alone driver:
    ./glsl_parser_test.py test_file.vert


=== Test Registration ===

Define a new function, add_glsl_parser_test(), which registers
the test by *only* specifying the test file. No longer does one
specify pass/fail and required GLSL version. For example:
    # file: /tests/all.tests
    add_glsl_parser_test(group, test_name, file_name)


=== References ===

[1] http://cgit.freedesktop.org/~chadversary/piglit/log/?h=glsl_parser_test

[2] git://anongit.freedesktop.org/~chadversary/piglit;
    cd piglit;
    git checkout glsl_parser_test

- --
Chad Versace
chad.versace at intel.com

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJM9D+fAAoJEAIvNt057x8igAsP/2TN0Hwf4+2++t4fjKehnEUx
9vBGtYdKljZAkwaiq5N+opkwZ7OzT2oKQ2GeIE/16y+HTzOyIQi4HEImapnOCuRz
O30ZKYYjw/3qh8DkZOOAzY1pzi4xQ533oUium+wpBbaJ7nNvMl76C4nhfgjT2oON
goWpk7yKqxYhK8UbG8IOZuHHsLZ9wsQp6ksC3tvntRmAjqLpKfjbbJKFL6JKur04
FuOxdIYe3OykHpxGvhVTaVWlT6WJ08uYMg9I1/wAdrMchfSsBO75fQwHoTfQHZdV
6EiuIFZiXuHU3qjpqqQxrGvkcNSon8jPIw28WLUudeEoD8hvznET47ob9xNAJVxC
P9Uz+JgEgm3lA1Bc6VknjTeXXTrUclN8PrdOpM9MGoDWxG/aTMvFyWUYTs0PtbJ7
QmI50A7BZJib0yC267d+fwftRQ/znFxaNUx+i2MBz5iMCFg3evDt7cZ+e7IinvOR
/QhIhRC1bBuCgARKK/v0/fhda1OneGnDgh6df8GTfWqMYMnfHkeriUV0qHcvSRrx
2AD5GXS3R1SMYCT3Z6+26ieZ6INhEHkqU/rhMaaq/X/lMZMDKyzigFwVjb78tmb+
gu5Z4Q0zAfsb9O6pLfl3yqJRAPSV1VNIKvUkm0DwBD9zrRv7lQPoSVrkuM39dsJ+
vBLW5E/sM1d3OiGcJkud
=wcMB
-----END PGP SIGNATURE-----