[Mesa-dev] [PATCH 03/16] mesa: Clamps the stencil value masks to GLint when queried

[Olivier Galibert]

Hmmmm, if you convert to float you have a real problem: floats only
have 23 bits of mantissa, so if bit 31 or 32 is set bits 0-7 will be
lost.  Converting directly won't change a thing there.  Initing to 255
is definitively better it seems.

W.r.t clamping, in computer graphics clamping a value to an interval
mean setting the value to the nearest boundary if it was outside of
the interval.  Clamping can never change a value to something *inside*
the interval, which masking does.

  OG.


On Thu, Dec 18, 2014 at 12:08 PM, Eduardo Lima Mitev <elima at igalia.com> wrote:
> On 12/18/2014 10:28 AM, Eduardo Lima Mitev wrote:
>> On 12/18/2014 09:55 AM, Olivier Galibert wrote:
>>> Something is not clear to me: In which way -1 is incorrect?
>>>
>>
>> Hi Olivier,
>>
>> The values being queried are the front and back stencil masks. Masks are
>> (conceptually?) an unsigned integer, AFAIU.
>
> Well, more accurately, just a string of bits, so -1 (or any other value)
> is probably fine. Problem is when the signed integer value is further
> converted to float, which is the case for these failing tests. (Note
> that only the test cases that query the mask as a float value are the
> ones failing).
>
> Giving a bit more of thought to this, and assuming the test is fine
> querying for a mask value using the glGetFloat API, the problem is Mesa
> converting from unsigned int (the mask) to signed int, then to float;
> instead of converting to float directly. I don't have a say on why it
> does that intermediate conversion to int, though.
>
> So my original solution was wrong from different angles, and the final
> solution is probably not the best one either, since we are "avoiding"
> the type conversion problem rather than fixing it.
>
> Thanks for rising these points.
>
> Eduardo
> _______________________________________________
> mesa-dev mailing list
> mesa-dev at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/mesa-dev