[Piglit] [PATCH v3 0/3] calculation of tolerance for complex GLSL built-ins

[Micah Fedke]

This series upgrades the ARB_shader_precision test generator with 
support for complex function tolerances.

The ARB_shader_precision spec does not give specific guidance in regard 
to acceptable error ranges for complex functions, supplying only the 
murky "Built-in functions defined in the specification with an equation 
built from the above operations inherit the above errors."  Currently, 
the shader_precision tests allow zero tolerance for complex functions, 
but this is causing many false negatives and needs to be improved.

This series is my best attempt at due diligence - any discussion/input 
is certainly welcomed

The theory behind this is probably worth a blog post and I won't belabor 
the point here, but this series basically implements a filter for the 
test_suite dict that will drop any test vectors that would push a 
function's output into regions of greatly multiplied error - the 
"badlands".  If the output does not trespass on the badlands, then the 
error estimate in ulps is calculated and attached to the current test 
vector for the shader_test to use as the allowable error range when 
analyzing the output generated on the GPU during test run-time.

The bigfloat package is a necessary dependency for this filter, as it 
provides both arbitrary-size floating point operations and it allows all 
floating point values and operations in a code block to be forced into a 
particular precision, leaving no question as to what precision was 
employed for an intermediate calculation.  This comes in handy when 
running the reference versions of the complex built-ins.  Performance 
impact is small (~6s for the entire gen_shader_precision_tests.py script 
on IvyBridge i7 @ 2.9GHz) and is localized to build-time.

The technique described above actually takes very little python code to 
construct - most of the complexity is in dealing with the polymorphic 
nature of the GLSL built-ins - componentwise vs. non-componentwise 
parameters, etc.


Micah Fedke (3):
arb_shader_precision: support scalar values in shader_runner_format
arb_shader_precision: add framework for calculating tolerances for 
complex functions
arb_shader_precision: enable calculation of complex function tolerances

  CMakeLists.txt                                               |   1 +
  cmake/Modules/FindPythonBigfloat.cmake                       |  43 
+++++++++
  generated_tests/gen_shader_precision_tests.py                | 650 
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------
  generated_tests/templates/gen_shader_precision_tests/fs.mako |  51 
++++++----
  generated_tests/templates/gen_shader_precision_tests/gs.mako |  51 
++++++----
  generated_tests/templates/gen_shader_precision_tests/vs.mako |  51 
++++++----
  6 files changed, 765 insertions(+), 82 deletions(-)


Changes since original:

Review comments addressed
- explicitly cast literals to BigFloats
- updated high_precision and low_precision for more clarity (use native 
precisions)
- removed _floatToBits and _bitsToFloat as they are no longer used
- clarified _ulpsize return
- renamed _sanitize_arg to _listify
- renamed _to_bigfloat_list to _to_bigfloat
- added an assert to _to_bigfloat to ensure proper types are passed in, 
and replaced double-map with list generator in return value
- commented all reference function with descriptions, inputs and outputs
- replaced non-componentwise arg array generation for loop with a map



Public repo:
http://cgit.collabora.com/git/user/fedke.m/piglit.git/log/?h=complex_tolerances_v3

-- 

Micah Fedke
Collabora Ltd.
+44 1223 362967
https://www.collabora.com/
https://twitter.com/collaboraltd