[Mesa-dev] Numeric precision problem with highp tanh

[Haixia Shi]

We're encountering failures in the latest version of dEQP (specifically,
dEQP-GLES3.functional.shaders.builtin_functions.precision.tanh.highp_*) on
all Intel devices.

The problem is that when the abs value of input is too large (say 80), the
function should return +/-1, but the actual shader output is 0.

The following patch in glsl compiler fixes the problem, but I'm not really
sure if a fix in the compiler frontend is the right place. On the other
hand, once the intermediate code is emitted, it's not really feasible for
drivers to handle precision problems in backend. Any ideas?

Here's the patch

diff --git a/src/compiler/glsl/builtin_functions.cpp
b/src/compiler/glsl/builtin_functions.cpp
index 3e4bcbb..6cfeb1b 100644
--- a/src/compiler/glsl/builtin_functions.cpp
+++ b/src/compiler/glsl/builtin_functions.cpp
@@ -3563,9 +3563,13 @@ builtin_builder::_tanh(const glsl_type *type)
    ir_variable *x = in_var(type, "x");
    MAKE_SIG(type, v130, 1, x);

+   /* clamp x to [-20, +20] to avoid numeric problems */
+   ir_variable *t = body.make_temp(type, "tmp");
+   body.emit(assign(t, min2(max2(x, imm(-20.0f)), imm(20.0f))));
+
    /* (e^x - e^(-x)) / (e^x + e^(-x)) */
-   body.emit(ret(div(sub(exp(x), exp(neg(x))),
-                     add(exp(x), exp(neg(x))))));
+   body.emit(ret(div(sub(exp(t), exp(neg(t))),
+                     add(exp(t), exp(neg(t))))));

    return sig;
 }
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/mesa-dev/attachments/20161208/c0d366c1/attachment.html>