[Mesa-dev] Gallium pixel formats on big-endian

[Michel Dänzer]

On Mon, 2013-01-28 at 06:56 -0500, Adam Jackson wrote: 
> I've been looking at untangling the pixel format code for big-endian.
> My current theory is that blindly byte-swapping values is just wrong.

Certainly. :) I think you're discovering that this hasn't really been
thought through beyond what's necessary for things to work with little
endian CPU and GPU. Any code there is for dealing with big endian CPUs
has been bolted on as an afterthought.

> Data coming from the application is never in the "wrong" order.
> Consider the first non-trivial format emitted in u_format_table.c:
> 
>     /* taken from an x86 build */
>     union util_format_b8g8r8a8_unorm {
>        uint32_t value;
>        struct {
>           uint8_t b;
>           uint8_t g;
>           uint8_t r;
>           uint8_t a;
>        } chan;
>     };
> 
> With little-endian struct packing, 'b' is the least significant byte and
> 'a' is the most significant.  Therefore the "format name" matches the
> memory order representation of the pixel format: b8g8r8a8 means eight
> bits of blue in address 0.  I believe this is the intent, and that this
> should be preserved independently of either the bit order in the
> channels themselves, or the byte order of the host.  Blue always goes at
> the low address if B is the first letter in the format, regardless of
> how wide B is or what CPU you're on.
> 
> When creating the unpack kernel on BE for this format, we byte-swap the
> value before channel extraction.  This happens to put the right channel
> values in the right bytes; but byte-swapping an r5g6b5 format, or any
> other format where channel boundaries are not byte boundaries, would be
> wrong.
> 
> I think the solution in this case is to simply emit format structs the
> other way around on big-endian:
> 
>     union util_format_b8g8r8a8_unorm {
>        uint32_t value;
>        struct {
>           uint8_t a;
>           uint8_t r;
>           uint8_t g;
>           uint8_t b;
>        } chan;
>     };

I'd rather argue the other way around: Formats such as r5g6b5 cannot be
defined in an endianness neutral manner, as components cross byte
boundaries. However, formats such as b8g8r8a8 can and possibly should be
defined as an array of bytes rather than as a packed value of any
particular endianness. So in your example above, the struct members
remain the same, but the 32 bit value (un)packing needs to take the CPU
endianness into account. (It might also be useful to allow describing
the same formats in both ways, to allow state trackers to choose the way
that better matches the API they implement)

There are other formats sort of in between, e.g. those with 16 or 32 bit
wide components. The individual components depend on endianness, but the
order of the components can be defined endianness neutrally according to
memory location.

For all cases where the format as a whole or its components depend on
endianness, there should probably be separate variants for little and
big endian packing, which could be mapped to native or foreign byte
order according to the CPU endianness where that makes sense.


> 2) Is there an additional (bit-)endianness assumption when talking to
> hardware?  For B5G6R5, is memory order:
> 
>     [b: 0-4] [g: 0-5] [r: 0-4]
> 
> or is it:
> 
>     [b: 4-0] [g: 5-0] [r: 4-0]

What's the difference? :)


-- 
Earthling Michel Dänzer           |                   http://www.amd.com
Libre software enthusiast         |          Debian, X and DRI developer