[Mesa-dev] [PATCH v3] compiler/glsl: fix precision problem of tanh

[Kenneth Graunke]

On Thursday, December 8, 2016 9:17:04 PM PST Jason Ekstrand wrote:
> On Thu, Dec 8, 2016 at 8:41 PM, Roland Scheidegger <sroland at vmware.com>
> wrote:
> 
> > I'm wondering, isn't that actually a problem of the test, that is it
> > can't actually expect reasonable results with such input values?
> > Since within the shader languages those functions which are composed of
> > multiple other functions are usually allowed to basically accumulate all
> > the errors of said functions. Though I agree that results outside [-1,1]
> > would be odd...
> >
> 
> No, not really.  tanh() is well defined on the entire real line and always
> stays inside the interval (-1, 1).  The problem is just that floating-point
> arithmatic explodes once x gets large enough.  However, long before that
> point, it's flattened out to +- 1 (Not mathematically, but as far as
> floating-point precision is concerned).
> 
> 
> > btw I'm wondering if some vendors wouldn't implement that with slightly
> > simplified formula, e.g. (e^2x - 1) / (e^2x + 1) (this is what nvidia
> > used for cg apparently according to docs, saving one of the
> > exponentials). Might be worse for accuracy though (and won't solve this
> > problem, though it would now only need a one-sided clamp).
> >
> 
> I would be interested to know if that formula would pass the dEQP precision
> tests...  It would be simpler.

It does, if you also include Haixia's clamping code.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part.
URL: <https://lists.freedesktop.org/archives/mesa-dev/attachments/20161208/90186ecf/attachment-0001.sig>