[Piglit] Fwd: [PATCH 1/1] RFC: add ability to report gl api supported by tests

[Tom Gall]

On Wed, Jan 30, 2013 at 3:25 PM, Eric Anholt <eric at anholt.net> wrote:
> Tom Gall <tom.gall at linaro.org> writes:
>
>> Why is this is a good patch to add?
>>
>> In the not too distant future piglit tests will be able to be built
>> once and at runtime one cam choose to test against api family, IE GL,
>> GLES or both. I further assert that one will be able to specify what
>> context versions should be tested, IE GLES1, GLES2, GLES3, all.
>>
>> This patch is a small part of these goals. When a testcase using this
>> code reports what it can do, it allows the python test harness to
>> decide if a testcase is a appropriate for running by asking it.
>
> If the testcase is not appropriate to run, the test just reports skip
> when you run it.  You're adding an extra invocation of the program for
> supported tests, and not saving any invocations of the program at all.

Obviously the patch as it stands doesn't touch the python
infrastructure to gather what a test supports or use it as a basis for
formulating the population of what should be run across API or context
versions. That's a future patch and I figured this patch in RFC form
might broach that topic.

I do like the idea of not having all.tests and I see this is a step in
that direction.

Ergo, why not do a find for the binaries, do a find for the various
verts, geo, frags etc, and then run across the population of what's
available based on what the user wants.

So when/if there is agreement that this is sane, my intent is to first
query the tests as they are found, then run it across the derivations
of API/content version depending on user input.

So a pattern might be for piglit-run.py -api=gl,gles2,gles3
for all tests
   test-foo -supports
   if supported test-foo  for gl4.3
   if supported test-foo  for gles2
   if supported test-foo  for gles3
move on to next test

Obviously the cache will be hot so adding one invocation in what will
be likely several successive invocations of the actual test after that
shouldn't much matter.

Granted one could take the position that all api tests should be run
for a particular context before moving on to the next api/context
variation.   IE run all gl before moving on to gles.


Slightly different topic, following your logic above and presuming
we're in a universe where we build the test but once, how did you
envision that a test will know to run against a particular api (gl,
gles) and context version?

Another variation on this might be to modify the framework in
tests/util so that when the test starts, it would run across the
api/version variations within the test itself. This seemed ugly as
compared to keeping loop & filter logic in the python code. Besides
avoiding bugs where teardown was less than perfect is probably a good
thing.

--
Regards,
Tom

"Where's the kaboom!? There was supposed to be an earth-shattering
kaboom!" Marvin Martian
Tech Lead, Graphics Working Group | Linaro.org │ Open source software
for ARM SoCs
w) tom.gall att linaro.org
h) tom_gall att mac.com