[Piglit] piglit-dispatch for gles

[Tom Gall]

On Thu, Jan 3, 2013 at 12:48 PM, Chad Versace
<chad.versace at linux.intel.com> wrote:
> On 01/03/2013 10:00 AM, Tom Gall wrote:
>
>> I'm still pulling apart how what is currently there works but all in
>> all seems straight forward and I'm not finding any pixie dust or
>> unicorn tears.
>>
>> My incomplete list, trying to break the problem apart, I had:
>> 0) add param to piglit-run.py to define which api / version
>> combinations should be run
>>   -)  in python there needs to be a way to to define a test as valid
>> for combinations of api / version. Just trying everything blindly
>> seems wrong.
>
> Yes, we definitely need something like that. But I think the metadata
> should go into the testfile itself.

I agree.

> Piglit's history has proved it unwise to maintain much test metadata outside
> of the testfile and in the python. The python scripts use to contain much more
> metadata. For example, the GLSL version and expected result for glslparser tests
> were kept in all.tests (see commits 6c7d874b and 71bb5333). The problem was
> that the metadata sometimes drifted out-of-sync with the actual test.
>
> One requirement for encoding the metadata in the testfile is that there must
> be a method for the python scripts to easily extract that metadata.
>
> I don't have a firm plan yet on how to accomplish that, but I do have tentative, messy
> plan. As the final step of the build phase, the python scripts would run each executable
> with a special flag, --give-me-json-that-tells-me-what-gl-flavors-you-support, and
> the scripts would cache the result. Then, when you run piglit-run.py, it would refer to
> the cached json in order to determine which set of gl flavors over which to iterate the
> test.
>
> That plan isn't perfect, but after much discussion with Ken Graunke and Paul,
> we were unable to formulate a better one. Any suggestions?

What if instead of parsing over the c source file, why not run the
test with a --tell-me-what-you-support sort of flag?

Spit out bog standard json in response and that's that.

ex:

minmax --q-support

returns something like:
{
    "minmax": {
        "api": [ 'gl', 'gles'],
        "gl": [1.0, 3.2],
        "gles": [2.0, 3.0]
    }
}

etc etc. It could also include other information that might be useful
for python, like say the category so even tho shader tests might be
spread all over, the reporting could keep them straight. Go way
overboard and one could include a list of what APIs are tested so
future driver writes when looking for tests that exercise foo have
something they could query for.

Obviously the python classes for shader_runner and glslparser already
parse the data files so would be fairly trivial to fix them up or
change glslparsertest and shader_runner so they did the parsing
instead of the python.

ex:

shader_runner --q-support path/to/shader_test

returns the support data from the test data file.

<snip>
>
> Even though there may exist many GL/GLES tests that could fall into the
> api-specific category, for simplicity we should probably place all GL/GLES
> into the build-once-and-dispatch-dynamically catgeory. I think that the only
> tests that should really go into the api-specific category are those that
> test windowing api's, such as the GLX and EGL tests.

In theory I agree. I've spent a bit of time looking at candidates for
porting over to gles or at least seeing what either might be quite
trivial to get working on gles.

There's common pitfalls that will hinder a build once and work
everywhere mantra for GL/GLES beyond missing APIs. For instance, glsl
vs glsl-es differences like no gl_Vertex, gl_ForegroundColor etc.
Fairly simple to deal with, but a pain.

Obviously one could preparse and replace problems like that on the fly
for glsl-es but such a contraption might be slightly aggravating to
maintain. :-)

It can all be worked though. Establishing conventions will help a great deal.

--
Regards,
Tom

"Where's the kaboom!? There was supposed to be an earth-shattering
kaboom!" Marvin Martian
Graphics Working Group | Linaro.org │ Open source software for ARM SoCs
w) tom.gall att linaro.org
h) tom_gall att mac.com