[PATCH] drm/amd/display: Fix pageflipping for XOrg in Linux 5.11+

[Bas Nieuwenhuizen]

I think the problem here is that application A can set the FB and then
application B can use getfb2 (say ffmpeg).

https://lists.freedesktop.org/archives/dri-devel/2021-January/292761.html
would be my alternative patch.

(I'm not good at detecting the effects of tearing  apparently but
tested this avoids the pageflip failure by debug-prints)

On Thu, Dec 31, 2020 at 9:52 PM Mario Kleiner
<mario.kleiner.de at gmail.com> wrote:
>
> Commit 816853f9dc4057b6c7ee3c45ca9bd5905 ("drm/amd/display: Set new
> format info for converted metadata.") may fix the getfb2 ioctl, but
> in exchange it completely breaks all pageflipping for classic user
> space, e.g., XOrg, as tested with both amdgpu-ddx and modesetting-ddx.
> This leads to massive tearing, broken visual timing/timestamping etc.
>
> Reason is that the classic pageflip ioctl doesn't allow a fb format
> change during flip, and at least X uses classic pageflip ioctl and no
> atomic modesetting api at all.
>
> As one attempted workaround, only set the new format info for converted
> metadata if the calling client isn't X. Not sure if this is the best
> way, or if a better check would not be "not all atomic clients" or
> similar? In any case it works for XOrg X-Server. Checking the ddx
> code of intel-ddx/modesetting-ddx/amdgpu-ddx as well as grepping over
> Mesa doesn't show any users of the getfb2 ioctl(), so the need for this
> format info assignment seems to be more the exception than the rule?
>
> Fixes: 816853f9dc40 ("drm/amd/display: Set new format info for converted metadata.")
> Cc: Bas Nieuwenhuizen <bas at basnieuwenhuizen.nl>
> Cc: Alex Deucher <alexander.deucher at amd.com>
> Signed-off-by: Mario Kleiner <mario.kleiner.de at gmail.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_display.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c
> index f764803c53a4..cb414b3d327a 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c
> @@ -828,7 +828,8 @@ static int convert_tiling_flags_to_modifier(struct amdgpu_framebuffer *afb)
>                         if (!format_info)
>                                 return -EINVAL;
>
> -                       afb->base.format = format_info;
> +                       if (afb->base.comm[0] != 'X')
> +                               afb->base.format = format_info;
>                 }
>         }
>
> --
> 2.25.1
>