Mesa (master): compiler/glsl: fix precision problem of tanh

[Jason Ekstrand]

Module: Mesa
Branch: master
Commit: d4983390a869c3051929858a8b783be53d46b722
URL:    http://cgit.freedesktop.org/mesa/mesa/commit/?id=d4983390a869c3051929858a8b783be53d46b722

Author: Haixia Shi <hshi at chromium.org>
Date:   Thu Dec  8 17:41:02 2016 -0800

compiler/glsl: fix precision problem of tanh

Clamp input scalar value to range [-10, +10] to avoid precision problems
when the absolute value of input is too large.

Fixes dEQP-GLES3.functional.shaders.builtin_functions.precision.tanh.* test
failures.

v2: added more explanation in the comment.
v3: fixed a typo in the comment.

Signed-off-by: Haixia Shi <hshi at chromium.org>
Reviewed-by: Jason Ekstrand <jason at jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth at whitecape.org>
Cc: "13.0" <mesa-dev at lists.freedesktop.org>

---

 src/compiler/glsl/builtin_functions.cpp | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/src/compiler/glsl/builtin_functions.cpp b/src/compiler/glsl/builtin_functions.cpp
index 3e4bcbb..3dead1a 100644
--- a/src/compiler/glsl/builtin_functions.cpp
+++ b/src/compiler/glsl/builtin_functions.cpp
@@ -3563,9 +3563,17 @@ builtin_builder::_tanh(const glsl_type *type)
    ir_variable *x = in_var(type, "x");
    MAKE_SIG(type, v130, 1, x);
 
+   /* Clamp x to [-10, +10] to avoid precision problems.
+    * When x > 10, e^(-x) is so small relative to e^x that it gets flushed to
+    * zero in the computation e^x + e^(-x). The same happens in the other
+    * direction when x < -10.
+    */
+   ir_variable *t = body.make_temp(type, "tmp");
+   body.emit(assign(t, min2(max2(x, imm(-10.0f)), imm(10.0f))));
+
    /* (e^x - e^(-x)) / (e^x + e^(-x)) */
-   body.emit(ret(div(sub(exp(x), exp(neg(x))),
-                     add(exp(x), exp(neg(x))))));
+   body.emit(ret(div(sub(exp(t), exp(neg(t))),
+                     add(exp(t), exp(neg(t))))));
 
    return sig;
 }