[PATCH] drm: fourcc byteorder: brings header file comments in line with reality.

[Ilia Mirkin]

On Sat, Apr 22, 2017 at 9:40 AM, Ilia Mirkin <imirkin at alum.mit.edu> wrote:
> On Sat, Apr 22, 2017 at 5:50 AM, Ville Syrjälä
> <ville.syrjala at linux.intel.com> wrote:
>> On Sat, Apr 22, 2017 at 01:07:57AM -0400, Ilia Mirkin wrote:
>>> On Fri, Apr 21, 2017 at 12:59 PM, Ville Syrjälä
>>> <ville.syrjala at linux.intel.com> wrote:
>>> > On Fri, Apr 21, 2017 at 10:49:49AM -0400, Ilia Mirkin wrote:
>>> >> On Fri, Apr 21, 2017 at 3:58 AM, Gerd Hoffmann <kraxel at redhat.com> wrote:
>>> >> > While working on graphics support for virtual machines on ppc64 (which
>>> >> > exists in both little and big endian variants) I've figured the comments
>>> >> > for various drm fourcc formats in the header file don't match reality.
>>> >> >
>>> >> > Comments says the RGB formats are little endian, but in practice they
>>> >> > are native endian.  Look at the drm_mode_legacy_fb_format() helper.  It
>>> >> > maps -- for example -- bpp/depth 32/24 to DRM_FORMAT_XRGB8888, no matter
>>> >> > whenever the machine is little endian or big endian.  The users of this
>>> >> > function (fbdev emulation, DRM_IOCTL_MODE_ADDFB) expect the framebuffer
>>> >> > is native endian, not little endian.  Most userspace also operates on
>>> >> > native endian only.
>>> >> >
>>> >> > So, go update the comments for all 16+24+32 bpp RGB formats.
>>> >> >
>>> >> > Leaving the yuv formats as-is.  I have no idea if and how those are used
>>> >> > on bigendian machines.
>>> >>
>>> >> I think this is premature. The current situation is that I can't get
>>> >> modetest to work *at all* on my NV34 / BE setup (I mean, it runs, just
>>> >> the colors displayed are wrong). I believe that currently it packs
>>> >> things in "cpu native endian". I've tried futzing with that without
>>> >> much success, although I didn't spend too much time on it. I have a
>>> >> NV34 plugged into my LE setup as well although I haven't tested to
>>> >> double-check that it all works there. However I'm quite sure it used
>>> >> to, as I used modetest to help develop the YUV overlay support for
>>> >> those GPUs.
>>> >
>>> > I just took a quick stab at fixing modetest to respect the current
>>> > wording in drm_fourcc.h:
>>> >
>>> > git://github.com/vsyrjala/libdrm.git modetest_endian
>>>
>>> Looks like there was some careless testing on my part :( So ... it
>>> looks like the current modetest without those changes does, in fact,
>>> work on NV34/BE. With the changes, it breaks (and the handling of the
>>> b* modes is a little broken in those patches -- they're not selectable
>>> from the cmdline.) Which means that, as Michel & co predicted, it
>>> appears to be taking BE input not LE input. This is very surprising to
>>> me, but it is what it is. As I mentioned before, the details of how
>>> the "BE" mode works on the GPUs is largely unknown to us beyond a few
>>> basics. Note that only XR24 works, AR24 ends up with all black
>>> displayed. This also happens on LE.
>>
>> Did you try 8bpp or 16bpp formats? I expect that if you've just blindly
>> enabled some magic byte swapper in the hardware it will only for
>> a specific pixel size.
>
> Thankfully dispnv04 exposes no such madness - just XR24 (and AR24,
> although that doesn't appear functional). Yes, it's likely that
> there's a byteswap happening somewhere. In fact the copy engines have
> parameters somewhere to tell how the swap should be done (basically
> what the element size is). I don't quite know how to set that though
> on this generation. I should poke at VRAM via the mmio peephole and
> see what's actually being stored. Although of course MMIO accesses are
> also auto-byteswapped. It's all just one big massive headache.

Or it could just be the obvious:

static void nv_crtc_commit(struct drm_crtc *crtc)
{
        struct drm_device *dev = crtc->dev;
        const struct drm_crtc_helper_funcs *funcs = crtc->helper_private;
        struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc);

        nouveau_hw_load_state(dev, nv_crtc->index,
&nv04_display(dev)->mode_reg);
        nv04_crtc_mode_set_base(crtc, crtc->x, crtc->y, NULL);

#ifdef __BIG_ENDIAN
        /* turn on LFB swapping */
        {
                uint8_t tmp = NVReadVgaCrtc(dev, nv_crtc->index,
NV_CIO_CRE_RCR);
                tmp |= MASK(NV_CIO_CRE_RCR_ENDIAN_BIG);
                NVWriteVgaCrtc(dev, nv_crtc->index, NV_CIO_CRE_RCR, tmp);
        }
#endif

        funcs->dpms(crtc, DRM_MODE_DPMS_ON);
        drm_crtc_vblank_on(crtc);
}

So presumably instead of that __BIG_ENDIAN thing, it should instead be
if crtc->primary->fb->fourcc & BIG_ENDIAN. (Although that's probably
going to break the universe.) There's also some questionable support
for 16-bit modes in the code, I'm going to see how easy it is to flip
on.

  -ilia