[Piglit] [PATCH 0/3] calculation of tolerance for complex GLSL built-ins

[Micah Fedke]

This series upgrades the ARB_shader_precision test generator with 
support for complex function tolerances.

The ARB_shader_precision spec does not give specific guidance in regard 
to acceptable error ranges for complex functions, supplying only the 
murky "Built-in functions defined in the specification with an equation 
built from the above operations inherit the above errors."  Currently, 
the shader_precision tests allow zero tolerance for complex functions, 
but this is causing many false negatives and needs to be improved.

This series is my best attempt at due diligence - any discussion/input 
is certainly welcomed

The theory behind this is probably worth a blog post and I won't belabor 
the point here, but this series basically implements a filter for the 
test_suite dict that will drop any test vectors that would push a 
function's output into regions of greatly multiplied error - the 
"badlands".  If the output does not trespass on the badlands, then the 
error estimate in ulps is calculated and attached to the current test 
vector for the shader_test to use as the allowable error range when 
analyzing the output generated on the GPU during test run-time.

The bigfloat package is a necessary dependency for this filter, as it 
provides both arbitrary-size floating point operations and it allows all 
floating point values and operations in a code block to be forced into a 
particular precision, leaving no question as to what precision was 
employed for an intermediate calculation.  This comes in handy when 
running the reference versions of the complex built-ins.  Performance 
impact is small (~6s for the entire gen_shader_precision_tests.py script 
on IvyBridge i7 @ 2.9GHz) and is localized to build-time.

The technique described above actually takes very little python code to 
construct - most of the complexity is in dealing with the polymorphic 
nature of the GLSL built-ins - componentwise vs. non-componentwise 
parameters, etc.


Micah Fedke (3):
arb_shader_precision: support scalar values in shader_runner_format
arb_shader_precision: add framework for calculating tolerances for 
complex functions
arb_shader_precision: enable calculation of complex function tolerances

  CMakeLists.txt                                               |   1 +
  cmake/Modules/FindPythonBigfloat.cmake                       |  43 
++++++++++++++
  generated_tests/gen_shader_precision_tests.py                | 402 
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------
  generated_tests/templates/gen_shader_precision_tests/fs.mako |  51 
++++++++++------
  generated_tests/templates/gen_shader_precision_tests/gs.mako |  51 
++++++++++------
  generated_tests/templates/gen_shader_precision_tests/vs.mako |  51 
++++++++++------
  6 files changed, 519 insertions(+), 80 deletions(-)


Changes since original:

Review comments addressed
- updated _is_sequence to use isinstance()
- simplified shader_runner_format(), added asserts
- added justification for bigfloat dep to commit msg
- added cmake check for bigfloat
- forced all args to be lists, to avoid scalar vs list madness
- removed ambiguous use of any()
- clarified the simple_fns vs complex_fns by moving mults into their own 
dict with associated logic updates and added a comment block
- eliminated dict of errors/badlands/component_tolerances, now just 
return the three as a tuple
- added comments to _gen_tolerance()
- added spacing around all math operators
- combined the framework and reference function commits, in order to 
avoid the cross-references between them


Public repo:
http://cgit.collabora.com/git/user/fedke.m/piglit.git/log/?h=complex_tolerances_v2

-- 

Micah Fedke
Collabora Ltd.
+44 1223 362967
https://www.collabora.com/
https://twitter.com/collaboraltd