[Piglit] [PATCH 0/1] Add comprehensive tests of builtin functions with uniform input.

[Ian Romanick]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/23/2011 08:32 AM, Eric Anholt wrote:
> On Sat, 23 Jul 2011 01:45:38 -0700, Ian Romanick <idr at freedesktop.org> wrote:
>> On 07/22/2011 02:00 PM, Paul Berry wrote:
>>> [require]
>>> GLSL >= 1.10
>>>
>>> [vertex shader]
>>> varying vec4 color;
>>> uniform float arg0;
>>> uniform float arg1;
>>> uniform vec4 arg2;
>>>
>>> void main()
>>> {
>>>   gl_Position = gl_Vertex;
>>>   vec4 result = smoothstep(arg0, arg1, arg2);
>>>   result -= -0.5;
>>>   result *= 0.5;
>>>   color = vec4(result);
>>> }
>>>
>>> [fragment shader]
>>> varying vec4 color;
>>>
>>> void main()
>>> {
>>>   gl_FragColor = color;
>>> }
>>>
>>> [test]
>>> uniform float arg0 -1.9
>>> uniform float arg1 -0.633333333333
>>> uniform vec4 arg2 -2.0 -0.666666666667 0.666666666667 2.0
>>> draw rect -1 -1 2 2
>>> probe rgba 0 0 0.25 0.748979443068 0.75 0.75
>>
>> Checks like this are a big problem for shaders.  With 8-bit color
>> outputs, checks that require any precision are doomed.  As you noticed,
>> you can only validate answers on the range [0,1].  Moreover, you can
>> only verify that the calculated answer is within 1/255 of correct.
>> That's just not good enough for most things.
>>
>> Most of the time you want to check that a calculation is correct within
>> a certain tolerance.  In those cases, you should do exactly that in the
>> shader.  Some of the non-constant indexing tests that I wrote do this.
>> In one of the cases, vs-varying-mat4-wr.shader_test, I want to check
>> that a matrix passed to fragment shader has a varying had correct
>> values.  I checked that by doing a calculation with that matrix and
>> verifying the correctness of that result:
>>
>> void main()
>> {
>>     gl_FragColor = (abs(distance(dst_matrix * v, expect)) < 1e-6)
>>         ? vec4(0.0, 1.0, 0.0, 1.0) : vec4(1.0, 0.0, 0.0, 1.0);
>> }
> 
> On the other hand, I was really happy to see that Paul did his tests
> this way.  I don't like the error tolerance stuff for driver debugging.
> I know, it makes sure that we're better than just 1/255 from the right
> answer, but when it comes to debugging all you get told is "pass" or
> "fail".  When you were debugging the matrix array access stuff, the bugs
> would have jumped out at us sooner to see a value smashed to 1.0 instead
> of approximately an expected color, or that just the zw channels were
> wrong.

In my case it was only the w... alpha... channel that was wrong.  I'm
not sure I would have noticed that. :)  But yeah, it doesn't give much
information to the person debugging the failure.

> Maybe we need to just be doing these tests into floating point FBOs?
> Then we could do error tolerance in our probes instead of the draw, not
> be limited in ranges, and not flash windows on the screen in -auto mode.
> And we could do fancy presentation based on the probe expected values
> for non-"-auto" mode.  Wouldn't that be awesome?

I think I'd opt for displaying both at once.  Use the red/green probe
for pass/fail, but also display the RGB representation for humans.
Being able to see one above the other would really make it pop, I think.

That would make things a fair amount more complicated.  The test would
either need:

  - Two different shaders, which shader_runner can't do.

  - A uniform to select the output mode.  This might force some tests to
run in software on some instruction-limited architectures.  I think we
can see this as an opportunity to improve code generation for those
architectures.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iEYEARECAAYFAk4tqRUACgkQX1gOwKyEAw/WeACfWkPUAAyeKmgXe/XkbhQntF+b
hsEAn2aLJNpgqC2n29QyivdrbUk6KUNe
=DtPG
-----END PGP SIGNATURE-----