[Mesa-dev] XCOM: Enemy Unknown vs. NaN texture unit LOD bias

[Roland Scheidegger]

Am 11.07.2017 um 08:25 schrieb Kenneth Graunke:
> Hello,
> 
> Mesa master has been hitting assert failures when running "XCOM: Enemy
> Unknown" since commit f8d69beed49c64f883bb8ffb28d4960306baf575, where we
> started asserting that the SAMPLER_STATE LOD Bias value actually fits in
> the correct number of bits.
> 
> Apparently, XCOM calls
> 
>    glTexEnv(GL_TEXTURE_FILTER_CONTROL_EXT, GL_TEXTURE_LOD_BIAS_EXT, val);
> 
> to set the texture unit LOD bias...but according to gdb, the value is:
> 
>    -nan(0x7ffff3)
> 
> In i965, we do CLAMP(lod bias, -16, 15)...but NaN ends up failing both
> the < min and > max comparisons, so it slips through.  But, that raises
> the question...what value *should* we be using?  0?  Min?  Max?
> 
> I couldn't find any immediately applicable GL spec text.  Anyone know of
> any?  If not, does DirectX mandate something?
I would guess behavior is undefined for GL.
I don't think d3d10 would say anything directly for this case, however
generally when things get converted to fixed point somewhere in the gpu
pipeline, the spec mandatates nans get converted to 0. This may or may
not be applicable here as it isn't really converted to fixed point
directly here.

We could also just make the CLAMP macro nan-safe easily (min2/max2
already are if you use them reversed...)
So instead of
#define CLAMP( X, MIN, MAX )  ( (X)<(MIN) ? (MIN) : ((X)>(MAX) ? (MAX) :
(X)) )

use
#define CLAMP( X, MIN, MAX ) ( (X)>(MIN) ? ((X)>(MAX) ? (MAX) : (X)) :
MIN) )

Which would fix this elsewhere too, with no extra effort even (at least
I'd think so - on x86_64, both should probably compile to just
minss/maxss combo, and if you get a nan or not there just depends on the
order of the arguments).
(This is, of course, assuming that min/max aren't NaN, but I'd think in
most (or all?) uses of CLAMP these are constant, non-NaN values).
Though it doesn't give you zero, unless the lower bound is zero (which
is probably often the case but not here). I have of course no idea if
the app would be happy with that...

And of course that's not filtering the values as they come through the api.

Roland


> 
> I wrote a hack to check isnan and replace it with 0, which gets the game
> working again, but...it seems like we could have this problem in a lot of
> other places too...and I'm not sure what the right answer is.
> 
> https://cgit.freedesktop.org/~kwg/mesa/commit/?h=xcom&id=6a1c0515b760c943eb547cced754b465aa3bd4ca
> 
> Thanks for any advice :)
> 
> --Ken
> 
> 
> 
> _______________________________________________
> mesa-dev mailing list
> mesa-dev at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
>