[PATCH 1/3] drm: fourcc byteorder: drop DRM_FORMAT_BIG_ENDIAN

[Ilia Mirkin]

On Tue, May 2, 2017 at 11:06 AM, Gerd Hoffmann <kraxel at redhat.com> wrote:
> Radeon and nvidia (nv40) cards where mentioned.  I'll try to summarize
> (feel free to correct me if I'm wrong).
>
> nvidia has support for 8 bit-per-color formats only on bigendian hosts.
> Not sure whenever this is a driver or hardware limitation.

Let me just summarize the NVIDIA situation. First off, pre-nv50 and
nv50+ are entirely different and unrelated beasts.

The (pre-nv50) hardware has (a few) "big endian mode" bits. Those bits
are kind of unrelated to each other and control their own "domains".
One of the domains is reading of the scanout fb. So as a result, the
hardware can scan out XRGB8888, RGB565, and XRGB1555 stored in either
little or big endian packings, irrespective of the "mode" that other
parts of the hardware are in.

However there's the delicate little question of the GPU *generating*
the data. These older GPUs don't have quite the format flexibility
offered by newer hw. So only XRGB8888 is supported, packed in whatever
"mode" the whole PGRAPH unit is in. (I say this because things seem to
work when rendering using the XRGB8888 format while scanning out with
the BE flag set.)

There are no APIs for controlling the endianness of each engine in
nouveau, so it ends up being in "big endian" mode on BE hosts, so the
GPU can only render to big-endian-packed framebuffers.

None of this applies to nv50+ hw. (Although it might in broad strokes.)

Currently the driver is exposing XRGB8888 and ARGB8888 formats as
that's what drm_crtc_init does for it. However the ARGB8888 format
doesn't work (and shouldn't be exposed, the alpha is meaningless on a
single-plane setup), and the XRGB8888 format is assumed to be packed
in cpu host endian (and the "BE" bit is set accordingly).

Hope this helps!

  -ilia