[PATCH v2 1/2] drm/virtio: Use XRGB8888 also for big endian systems

[Jocelyn Falempe]

On 21/08/2024 13:12, Gerd Hoffmann wrote:
> On Tue, Aug 20, 2024 at 11:07:40AM GMT, Jocelyn Falempe wrote:
>> Mesa doesn't support BGRX8888, that means most wayland compositors
>> don't work on big endian guests.
> 
> So you are doing a hard switch from native endian to little endian.
> 
> While this should be fine for modern userspace API (aka ADDFB2 ioctl) it
> is not for older APIs (ADDFB ioctl, also fbdev emulation) where only
> depth=32 is specified and userspace typically expects a framebuffer in
> native byte order.
> 
> Ideally virtio-gpu would support both big endian and little endian
> framebuffer formats (simliar to bochs drm driver).  That probably is a
> somewhat more invasive change because the DRM_IOCTL_MODE_CREATE_DUMB
> doesn't tell use the format which will be used.  Possible options I see:
> 
>    (1) Be lazy on creating host resources, i.e. call
>        virtio_gpu_cmd_create_resource() not at DRM_IOCTL_MODE_CREATE_DUMB
>        time but later when the resource will be actually be used (and
>        specifically after DRM_IOCTL_MODE_ADDFB(2) ioctl so we know the
>        format).  Needs additional state tracking (whenever the resource
>        has been created or not) in possibly lots of places.
> 
>    (2) Support changing the resource format, i.e. in case
>        DRM_IOCTL_MODE_ADDFB(2) is called with a format different from the
>        current one go through a destroy-and-recreate cycle for the host
>        resource.  Might have tricky corner cases (resource being in use
>        when DRM_IOCTL_MODE_ADDFB(2) is called).

I've implemented (1), I will send a new series soon.

Thanks for your advice.

-- 

Jocelyn

> 
> HTH & take care,
>    Gerd
>