[Mesa-dev] XCOM: Enemy Unknown vs. NaN texture unit LOD bias

[Ian Romanick]

On 07/10/2017 11:25 PM, Kenneth Graunke wrote:
> Hello,
> 
> Mesa master has been hitting assert failures when running "XCOM: Enemy
> Unknown" since commit f8d69beed49c64f883bb8ffb28d4960306baf575, where we
> started asserting that the SAMPLER_STATE LOD Bias value actually fits in
> the correct number of bits.
> 
> Apparently, XCOM calls
> 
>    glTexEnv(GL_TEXTURE_FILTER_CONTROL_EXT, GL_TEXTURE_LOD_BIAS_EXT, val);
> 
> to set the texture unit LOD bias...but according to gdb, the value is:
> 
>    -nan(0x7ffff3)
> 
> In i965, we do CLAMP(lod bias, -16, 15)...but NaN ends up failing both
> the < min and > max comparisons, so it slips through.  But, that raises
> the question...what value *should* we be using?  0?  Min?  Max?
> 
> I couldn't find any immediately applicable GL spec text.  Anyone know of
> any?  If not, does DirectX mandate something?

There is one place.  Section 2.3.4.1 (Floating-Point Computation) of the
OpenGL 4.5 core profile spec says:

   "The result of providing a value that is not a floating-point number
   to such a command is unspecified, but must not lead to GL
   interruption or termination. In IEEE arithmetic, for example,
   providing a negative zero or a denormalized number to a GL command
   yields predictable results, while providing a NaN or an infinity
   yields unspecified results."

Crashing is not allowed, but nearly any other behavior is.  Based on
that, I like Roland's suggestion of changing the CLAMP() macro.  We
should also report this to the developers.  I'd wager that some crazy
NaN value is not what they intended.

> I wrote a hack to check isnan and replace it with 0, which gets the game
> working again, but...it seems like we could have this problem in a lot of
> other places too...and I'm not sure what the right answer is.
> 
> https://cgit.freedesktop.org/~kwg/mesa/commit/?h=xcom&id=6a1c0515b760c943eb547cced754b465aa3bd4ca
> 
> Thanks for any advice :)
> 
> --Ken

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/mesa-dev/attachments/20170713/2f8b4b80/attachment.sig>