[Mesa-dev] [PATCH v3] compiler/glsl: fix precision problem of tanh

[Giuseppe Bilotta]

On Tue, Dec 20, 2016 at 2:17 AM, Matt Turner <mattst88 at gmail.com> wrote:
> On Mon, Dec 19, 2016 at 5:12 PM, Giuseppe Bilotta
> <giuseppe.bilotta at gmail.com> wrote:
>> Just one question though —not knowing much of the shader language, can
>> I expect expm1 to be available?
>
> No, expm1 doesn't exist in GLSL.

This is extremely bothersome. Both the (exp(2x)-1)/(exp(2x)+1) and the
1-2/(exp(2x)+1) formulas give pretty good results when written
in terms of expm1.

On Tue, Dec 20, 2016 at 3:48 AM, Roland Scheidegger <sroland at vmware.com> wrote:
> Not sure it really matters though one way or another. If you wanted good
> accuracy around 0, you'd have to use a different formula plus a select
> (seems like libm implementations actually use 3 cases depending on input
> value magnitude - not so hot with vectors, but thankfully glsl doesn't
> require 1 ULP accuracy).

Brute-forcing over all floating points on CPU by switching between the
two formulas above at appropriate thresholds gives a maximum relative
error of the order of machine epsilon when using expm1, and the switch
between the two formulas can be implemented with a select on two
terms. However, this does require expm1.

Nelson Beebe has a very detailed description of how to achieve very
accurate results for tanh here
https://www.math.utah.edu/~beebe/software/ieee/tanh.pdf and the
results are a bit depressing, in that multiple thresholds are
necessary. I'm not sure if these are the same used by libm, but in any
case neither lends itself well to vectorization (in contrast to the
switch between the two formulas above).

An alternative approach could be to actually provide a software
implementation of expm1 and use it to compute tanh. I wouldn't be
surprised if this would turn out to not be slower than using exp
itself, in fact.

-- 
Giuseppe "Oblomov" Bilotta