[Mesa-dev] [PATCH 1/2] mesa: remove buggy assertions in unpack Z24

[Brian Paul]

On Sun, Dec 11, 2011 at 8:46 AM, Marek Olšák <maraeo at gmail.com> wrote:
> On Mon, Nov 21, 2011 at 6:52 PM, Eric Anholt <eric at anholt.net> wrote:
>> On Sun, 20 Nov 2011 15:08:55 +0100, Marek Olšák <maraeo at gmail.com> wrote:
>>> unpack_float_z_Z24_X8 fails with -O2 in:
>>> - fbo-blit-d24s8
>>> - fbo-depth-sample-compare
>>> - fbo-readpixels-depth-formats
>>> - glean/depthStencil
>>>
>>> With -O0, it works fine.
>>>
>>> I am removing all the assertions. There's not much point in having them,
>>> is there?
>>
>> Is -ffast-math involved at all?
>
> Yes, -fno-fast-math makes the problem go away.
>
>>
>> At a guess, replacing "* scale" with "/ (float)0xffffff" makes the
>> failure happen either way?
>
> Yes. Only using GLdouble fixes it.
>
> It makes sense. The mantissa without the sign has 23 bits, but 24 bits
> are required to exactly represent 0xffffff if I am not mistaken.
>
> I have found a couple more cases where it could fail in the same way
> and fixed them in the attached patch. Please review.

Looks good.  Thanks for fixing this.

I guess it would have saved you some work if we'd have #defined some
Z24SCALE and Z32SCALE constants at the top of the file in the first
place...

Reviewed-by: Brian Paul <brianp at vmware.com>