[Mesa-dev] NaN behavior in GLSL (was Re: [PATCH] glsl: always do sqrt(abs()) and inversesqrt(abs()))

[Matteo Bruni]

2017-01-11 19:09 GMT+01:00 Jason Ekstrand <jason at jlekstrand.net>:
> Another reason why I'm not a huge fan is that there is some momentum in the
> industry to make GLSL better defined with respect to NaN.  I don't know that
> anything will ever come of it (because it may break apps) but if something
> does, we may find ourselves having to make SQRT and RSQ NaN-correct in the
> future and, hey look, it'll break apps.

Hijacking the thread to focus on this. What's the interest about
specifically (assuming that's something that can be talked about
publicly?)

I'm asking because in Wine we have been bitten many times by the GLSL
behavior WRT NaN and at the moment we really can't do anything
sensible about it in the general case. Many d3d9 applications depend
on not generating NaN in cases where e.g. IEEE 754 would expect to.
The RSQ (i.e. inversesqrt()) from the original thread is one of those,
but that's not really a problem for sane applications since it's
"defined" in d3d8/d3d9 as taking the absolute value of the operand and
indeed we translate RSQ to inversesqrt(abs(x)). The real issue is with
cases like 0*inf, which AFAIK on d3d9 are supposed to give 0 as
result.

Of course, that changes with d3d10+, which generally requires to
generate and preserve NaN, as Roland already mentioned.

So, what would be really nice to have is a GLSL extension for some
kind of switch to select the requested behavior WRT NaN. For example a
three-way option with "don't generate NaN in arithmetic operations",
"do generate NaN" and "don't care". It could also be a GL state if
that's easier to implement with the existing hardware, since an
individual application isn't supposed to require different behavior
from one shader to the next.

Is anyone interested in / favorable to something like this? It would
solve the issue with defining NaN behavior in GLSL while making things
a bit more compatible with "other API a lot of games are ported from
which happens to be supported by all the desktop GPUs".


Cheers,
Matteo Bruni.