[Piglit] [PATCH 0/1] Add comprehensive tests of builtin functions with uniform input.

[Ian Romanick]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/23/2011 06:41 AM, Paul Berry wrote:
> On 23 July 2011 01:45, Ian Romanick <idr at freedesktop.org> wrote:
>>> [test]
>>> uniform float arg0 -1.9
>>> uniform float arg1 -0.633333333333
>>> uniform vec4 arg2 -2.0 -0.666666666667 0.666666666667 2.0
>>> draw rect -1 -1 2 2
>>> probe rgba 0 0 0.25 0.748979443068 0.75 0.75
>>
>> Checks like this are a big problem for shaders.  With 8-bit color
>> outputs, checks that require any precision are doomed.  As you noticed,
>> you can only validate answers on the range [0,1].  Moreover, you can
>> only verify that the calculated answer is within 1/255 of correct.
>> That's just not good enough for most things.
> 
> That's a really good point.  As long as we are going to all this
> effort to comprehensively test all the built-in functions, we really
> should test them to a reasonable precision.
> 
>>
>> Most of the time you want to check that a calculation is correct within
>> a certain tolerance.  In those cases, you should do exactly that in the
>> shader.  Some of the non-constant indexing tests that I wrote do this.
>> In one of the cases, vs-varying-mat4-wr.shader_test, I want to check
>> that a matrix passed to fragment shader has a varying had correct
>> values.  I checked that by doing a calculation with that matrix and
>> verifying the correctness of that result:
>>
>> void main()
>> {
>>    gl_FragColor = (abs(distance(dst_matrix * v, expect)) < 1e-6)
>>        ? vec4(0.0, 1.0, 0.0, 1.0) : vec4(1.0, 0.0, 0.0, 1.0);
>> }
> 
> I think it would be fairly trivial to change
> gen_builtin_uniform_tests.py to generate code like this.  I'll submit
> a follow-up patch.
> 
>>
>> Of course, this assumes that abs() and distance() work correctly. :)
> 
> Actually the call to abs() could be dropped, since distance() always
> returns a nonnegative result.  But your point remains.  If you want to

Blarg.  That's from me spending too much time with other packages that
return a signed distance.  You are correct that the abs() can (should)
be dropped.

> be pedantic about it (and who doesn't?) we're also assuming that <,
> ?:, vec4(), gl_FragColor, and floating point constants work correctly
> :)
> 
> I think the right balance between caution and solipsism here is to
> drop the call to abs(), and do a little extra work in the distance()
> test so that it's not begging the question.  I'll try to do this in my
> follow-up patch.
> 
> In your test, how did you decide on 1e-6 as the tolerance?  According

I had been using 1e-6 as the tolerance for other (non-GLSL) things that
I have been doing.  It's not based on anything other than ensuring that
a usable result is produced.

> to GLSL 1.20 (in section 4.1.4 "Floats"), "It is not required that the
> precision of internal processing match the IEEE floating-point
> specification for floating-point operations, but the guidelines for
> precision established by the OpenGL 1.4 specification must be met."  I
> don't have the OpenGL 1.4 spec handy, but OpenGL 2.1 says (in section
> 2.1.1 "Floating-Point Computation") "We require simply that numbers'
> floating-point parts contain enough bits ... so that individual
> results of floating-point operations are accurate to about 1 part in
> 10^5."  I haven't checked whether later versions of OpenGL and GLSL
> impose stricter precision requirements.

GLSL 1.30+ require IEEE 754 single precision.  Later versions also add
requirements for the precision of individual operations (see below).

> It seems to me that this means we should use a tolerance of at least
> 1e-5, and sometimes more than that (in cases like degrees(), where the
> derivative of the output with respect to the input is substantially

1e-5 is probably a safe baseline.  We can then use
GL_ARB_shader_precision
(http://www.opengl.org/registry/specs/ARB/shader_precision.txt) to
provide some guidance about how much precision is lost by various
operations.

> greater than 1).  I'll mess around with builtin_function.py and see
> how difficult it would be to make it compute an appropriate tolerance
> for every test vector.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iEYEARECAAYFAk4tpkAACgkQX1gOwKyEAw9mBwCfUBtKN06mb4YI0i+YctSjRKQx
OeYAn3qUDzc90aUIWyRb1rw1uHOgP5AI
=oAZC
-----END PGP SIGNATURE-----