[Piglit] [RFC] One enum coverage test or many?

[Jose Fonseca]

----- Original Message -----
> I'm working on some generated "coverage" tests.  For each extension
> (or
> core GL version), these tests try every possible pname enum to the
> various get commands.  The tests just verify that no error is
> generated
> if the extension is supported or that INVALID_ENUM is generated if
> the
> extension is not supported.

I'm curious on how you're auto-generate this. apitrace the code to get a lot of state is autogenerated, but it ends up being interleaved with a lot of hand written code, as many kinds objects need to be bound/unbound, in order to see their state.

> Right now, a separate test is generated for each extension.  As a
> result, there are a bunch of ext_foo-get-enum-coverage tests.  I'm a
> bit
> torn between this implementation and one giant test that just pokes
> at
> everything.
> 
> Opinions?


Although I agree with others that having individual and properly categorized tests would be nice, I think that something that's easy to maintain beats something that looks good but is a burden to maintain. If you can successfully autogenerate most of this, then creating a monolithic executable that takes the extension as an argument seems a good compromise. Each extension can still be tested seperately and results put in the same group as the rest of the extension.

Jose