[Mesa-dev] [PATCH 1/2] mesa: remove buggy assertions in unpack Z24

[Marek Olšák]

On Mon, Nov 21, 2011 at 6:52 PM, Eric Anholt <eric at anholt.net> wrote:
> On Sun, 20 Nov 2011 15:08:55 +0100, Marek Olšák <maraeo at gmail.com> wrote:
>> unpack_float_z_Z24_X8 fails with -O2 in:
>> - fbo-blit-d24s8
>> - fbo-depth-sample-compare
>> - fbo-readpixels-depth-formats
>> - glean/depthStencil
>>
>> With -O0, it works fine.
>>
>> I am removing all the assertions. There's not much point in having them,
>> is there?
>
> Is -ffast-math involved at all?

Yes, -fno-fast-math makes the problem go away.

>
> At a guess, replacing "* scale" with "/ (float)0xffffff" makes the
> failure happen either way?

Yes. Only using GLdouble fixes it.

It makes sense. The mantissa without the sign has 23 bits, but 24 bits
are required to exactly represent 0xffffff if I am not mistaken.

I have found a couple more cases where it could fail in the same way
and fixed them in the attached patch. Please review.

Marek
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-mesa-fix-possible-precision-issues-in-pack-unpack-fe.patch
Type: text/x-diff
Size: 7218 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/mesa-dev/attachments/20111211/8d138367/attachment-0001.patch>