DRM_FORMAT_* byte order (was: Re: [PATCH] drm: virtio: fix virtio_gpu_cursor_formats)

[Ville Syrjälä]

On Fri, Apr 07, 2017 at 10:29:00AM +0200, Gerd Hoffmann wrote:
>   Hi,
> 
> > Hmm. Maybe it's still possible to salvage something by redefining the
> > BIG_ENDIAN format bit to mean the "the other endianness". Ugly but it
> > might still result in something usable.
> 
> Also at least for the virtual machine use case this doesn't buy us much.
> The drm drivers (at least the ones used on both big and little endian
> guests) support only 32 bpp + depth 24 formats.  And for these we don't
> need a "other endian" flag because we have fourcc codes for all sorts of
> byte orders (i.e. DRM_FORMAT_XRGB8888 little endian ==
> DRM_FORMAT_BGRX8888 big endian).

Yeah, those could be handled without the flag. But when mixed with any
other format the code would look a bit weird IMO. So my idea with the
flag was that if you display is big endian you always have the flag, and
then you don't have to think so much which way the bytes go for each
format. Less special casing is good IMO.

> 
> The DRM_FORMAT_BIG_ENDIAN flags also seems not be used anywhere in the
> code base (except in some format printing debug code ...).

That's because either no one has big endian display hardware or they
do but just didn't read the docs and cargo culted the format handling
from some driver for little endian display.

-- 
Ville Syrjälä
Intel OTC