[Mesa-dev] [PATCH 2/2] meta: Handle bitmaps with alpha test enabled.

[Brian Paul]

On 11/01/2010 08:02 PM, Francisco Jerez wrote:
> Brian Paul<brianp at vmware.com>  writes:
>
>> [...]
>>>
>>> -   bitmap8 = (GLubyte *) calloc(1, width * height);
>>> +   bitmap8 = (GLubyte *) malloc(width * height);
>>>      if (bitmap8) {
>>> +      memset(bitmap8, ~alpha, width * height);
>>>         _mesa_expand_bitmap(width, height,&unpackSave, bitmap1,
>>> -                          bitmap8, width, 0xff);
>>> +                          bitmap8, width, alpha);
>>>
>>>         _mesa_set_enable(ctx, tex->Target, GL_TRUE);
>>>
>>>         _mesa_set_enable(ctx, GL_ALPHA_TEST, GL_TRUE);
>>> -      _mesa_AlphaFunc(GL_GREATER, 0.0);
>>> +      _mesa_AlphaFunc(GL_EQUAL, ctx->Current.RasterColor[ACOMP]);
>>
>> I think the code in this area should stay as-is.  If the ubyte value
>> in the texture is converted to float, that might produce a value
>> that's different from the alpha you're specifying to
>> _mesa_AlphaFunc().  The test for equality could fail unexpectedly.
>>
>> The original code is more robust in that regard.
>
> What if the current raster alpha value is non-0xff? I changed the
> original code because that case was misrendered. I understand your
> concern though.

The original code checked if alpha test was enabled and fell back to 
swrast so the current alpha ref was irrelevant.  In the meta code 
which draws the bitmap with a textured quad, the current raster alpha 
color is also irrelevant since we're getting the fragment alpha value 
from the texture map.


> According to the spec both reference and fragment alpha values are
> converted to the nearest fixed-point value (of unspecified but same
> precision) before comparison, so AFAICT this change should cause no
> problems as long as we observe the same rules in the raster color to
> texture value conversion.

That part of the spec doesn't apply to floating point color buffers, 
BTW.  Also, when converting to floats to GLubyte (or GLushorts, etc) 
or vice versa we have to pay attention to whether doing it with the 
CPU will result in the same thing as the GPU.

As I wrote before, if the texture's ubyte alpha value is converted to 
float by the GPU and it compares it to the alpha ref value using 
floating point, testing for equality is tricky.  An inequality is safer.


> I'll reply to myself with a split version of this patch so that we can
> discuss both changes separately.

Good idea.


> I've made it use CLAMPED_FLOAT_TO_UBYTE
> as additional paranoia to avoid rounding errors in the alpha conversion.

I don't think that's sufficient.

-Brian