[PATCH] drm/fbdev-generic: Fix framebuffer on big endian devices

[Thomas Zimmermann]

Hi

Am 01.07.24 um 10:54 schrieb Geert Uytterhoeven:
> Hi Thomas,
>
> On Mon, Jul 1, 2024 at 10:42 AM Thomas Huth <thuth at redhat.com> wrote:
>> On 28/06/2024 08.07, Thomas Zimmermann wrote:
>>> Am 27.06.24 um 19:35 schrieb Thomas Huth:
>>>> Starting with kernel 6.7, the framebuffer text console is not working
>>>> anymore with the virtio-gpu device on s390x hosts. Such big endian fb
>>>> devices are usinga different pixel ordering than little endian devices,
>>>> e.g. DRM_FORMAT_BGRX8888 instead of DRM_FORMAT_XRGB8888.
>>>>
>>>> This used to work fine as long as drm_client_buffer_addfb() was still
>>>> calling drm_mode_addfb() which called drm_driver_legacy_fb_format()
>>>> internally to get the right format. But drm_client_buffer_addfb() has
>>>> recently been reworked to call drm_mode_addfb2() instead with the
>>>> format value that has been passed to it as a parameter (see commit
>>>> 6ae2ff23aa43 ("drm/client: Convert drm_client_buffer_addfb() to
>>>> drm_mode_addfb2()").
>>>>
>>>> That format parameter is determined in drm_fbdev_generic_helper_fb_probe()
>>>> via the drm_mode_legacy_fb_format() function - which only generates
>>>> formats suitable for little endian devices. So to fix this issue
>>>> switch to drm_driver_legacy_fb_format() here instead to take the
>>>> device endianness into consideration.
>>>>
>>>> Fixes: 6ae2ff23aa43 ("drm/client: Convert drm_client_buffer_addfb() to
>>>> drm_mode_addfb2()")
>>>> Closes: https://issues.redhat.com/browse/RHEL-45158
>>>> Signed-off-by: Thomas Huth <thuth at redhat.com>
>>> Acked-by: Thomas Zimmermann <tzimmermann at suse.de>
>>>
>>>
>>>> ---
>>>>    drivers/gpu/drm/drm_fbdev_generic.c | 3 ++-
>>> This file is now called drm_fbdev_ttm.c in drm-misc-next.
>> Oh, ok, shall I send a v2 that is adjusted to that change, or can it be
>> fixed while applying my patch?
> As this is a regression in mainline, which needs to be backported,
> too, it's best to apply your fix to v6.10-rc6, which does not have
> drm_fbdev_ttm.c yet.

I guess we can put the current fix into drm-misc-fixes.

>
>>> And a similar patch might be necessary for drm_fbdev_dma.c.
>> Looks similar, indeed. Shall I send a patch for that one, too? ... I
>> currently don't have a setup for testing that, though...

It's up to you. You'd need drm-misc-next from 
https://gitlab.freedesktop.org/drm/misc/kernel

Best regards
Thomas

> Obviously these need to be fixed, too.
>
> Gr{oetje,eeting}s,
>
>                          Geert
>

-- 
--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Frankenstrasse 146, 90461 Nuernberg, Germany
GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman
HRB 36809 (AG Nuernberg)