[Mesa-dev] [PATCH] mesa: adjust usage of swapBytes/littleEndian in format_matches_format_and_type

[Michel Dänzer]

On Don, 2013-01-31 at 17:06 +0100, Marek Olšák wrote: 
> - swapBytes has no effect on 8-bit single-component formats
> - GL_SHORT is in host byte order, so checking for littleEndian is unnecessary,
>   I decided to make the change for single-component formats only

Fair enough. BTW, the same is true for GL_INT. :)

> - for 8_8 formats, instead of littleEndian && !swapBytes,
>   use littleEndian != swapBytes to include big endian hosts
> 
> Based on suggestions from Michel Dänzer.

Thanks for tackling this!


>     case MESA_FORMAT_DUDV8:
>        return (format == GL_DU8DV8_ATI || format == GL_DUDV_ATI) &&
> -             type == GL_BYTE && littleEndian && !swapBytes;
> +             type == GL_BYTE && littleEndian != swapBytes;
[...] 
>     case MESA_FORMAT_SIGNED_RG88_REV:
> -      return format == GL_RG && type == GL_BYTE && littleEndian &&
> -             !swapBytes;
> +      return format == GL_RG && type == GL_BYTE && littleEndian != swapBytes;
[...] 
>     case MESA_FORMAT_SIGNED_AL88:
>        return format == GL_LUMINANCE_ALPHA && type == GL_BYTE &&
> -             littleEndian && !swapBytes;
> +             littleEndian != swapBytes;

I'm afraid these are not quite correct yet. I'm not 100% sure after
looking at the red book and the GL 3.x spec — this is head spinning
material, isn't it? :\ — but I think swapBytes only applies to the basic
type, in these cases GL_BYTE, so it's irrelevant. However, because the
Mesa formats are defined as 16 bit packed values in host byte order,
these cases can only match directly on little endian hosts, i.e. if
(type == GL_BYTE && littleEndian).


-- 
Earthling Michel Dänzer           |                   http://www.amd.com
Libre software enthusiast         |          Debian, X and DRI developer