[Mesa-dev] [PATCH v3] compiler/glsl: fix precision problem of tanh

[Giuseppe Bilotta]

Hello,

I realize that I'm a little late to comment about this, but I think
the formula used for
tanh should be changed again. Specifically, as suggested by Roland

On Fri, Dec 9, 2016 at 5:41 AM, Roland Scheidegger <sroland at vmware.com> wrote:
> btw I'm wondering if some vendors wouldn't implement that with slightly
> simplified formula, e.g. (e^2x - 1) / (e^2x + 1) (this is what nvidia
> used for cg apparently according to docs, saving one of the
> exponentials). Might be worse for accuracy though (and won't solve this
> problem, though it would now only need a one-sided clamp).

Another option is the 1 - 2/(1+expf(2x)), or even better 1 -
2/(2+expm1f(2x)).. I've run some tests and this seems to have the same
accuracy as the
one mentioned by Roland, with the bonus benefit of not needing any
clamping. The accuracy seems to actually be better
than the direct evaluation (difference over sum of exps), except
around 0 (say, when abs(x) < 1).

I've found the relative error away from 0 to be typically in the same
order of magnitude as the error in tanhf() itself (compared to tanh())
, and generally less than machine epsilon.. I'm currently looking at
options to improve the accuracy without clamping and without excessive
additional computations, might propose a patch in the next couple of
days.

Just one question though —not knowing much of the shader language, can
I expect expm1 to be available?

-- 
Giuseppe "Oblomov" Bilotta