[Mesa-dev] XCOM: Enemy Unknown vs. NaN texture unit LOD bias

[Marek Olšák]

On Tue, Jul 11, 2017 at 3:08 PM, Roland Scheidegger <sroland at vmware.com> wrote:
> Am 11.07.2017 um 08:25 schrieb Kenneth Graunke:
>> Hello,
>>
>> Mesa master has been hitting assert failures when running "XCOM: Enemy
>> Unknown" since commit f8d69beed49c64f883bb8ffb28d4960306baf575, where we
>> started asserting that the SAMPLER_STATE LOD Bias value actually fits in
>> the correct number of bits.
>>
>> Apparently, XCOM calls
>>
>>    glTexEnv(GL_TEXTURE_FILTER_CONTROL_EXT, GL_TEXTURE_LOD_BIAS_EXT, val);
>>
>> to set the texture unit LOD bias...but according to gdb, the value is:
>>
>>    -nan(0x7ffff3)
>>
>> In i965, we do CLAMP(lod bias, -16, 15)...but NaN ends up failing both
>> the < min and > max comparisons, so it slips through.  But, that raises
>> the question...what value *should* we be using?  0?  Min?  Max?
>>
>> I couldn't find any immediately applicable GL spec text.  Anyone know of
>> any?  If not, does DirectX mandate something?
> I would guess behavior is undefined for GL.
> I don't think d3d10 would say anything directly for this case, however
> generally when things get converted to fixed point somewhere in the gpu
> pipeline, the spec mandatates nans get converted to 0. This may or may
> not be applicable here as it isn't really converted to fixed point
> directly here.
>
> We could also just make the CLAMP macro nan-safe easily (min2/max2
> already are if you use them reversed...)
> So instead of
> #define CLAMP( X, MIN, MAX )  ( (X)<(MIN) ? (MIN) : ((X)>(MAX) ? (MAX) :
> (X)) )
>
> use
> #define CLAMP( X, MIN, MAX ) ( (X)>(MIN) ? ((X)>(MAX) ? (MAX) : (X)) :
> MIN) )

I like this. It's certainly the cheapest way of converting NaNs to MIN.

Marek