[PATCH] drm: Allow format change during legacy page-flip if driver is atomic.

[Mario Kleiner]

This is a slight improvement for legacy flipping, but also an attempted
fix for a bug/regression introduced into Linux 4.11-rc.

Commit 816853f9dc4057b6c7ee3c45ca9bd5905 ("drm/amd/display: Set new
format info for converted metadata.") fixes the getfb2 ioctl, but
in exchange it completely breaks all pageflipping for classic user
space, e.g., XOrg, as tested with both amdgpu-ddx and modesetting-ddx.
This leads to massive tearing, broken visual timing/timestamping, and
a xorg log flooded with error messages, as tested on Ubuntu 20.04.1-LTS
with X-Server 1.20.8, Mesa 20.0.8, amdgpu-ddx 19.1.0 and also with the
modesetting-ddx with/without atomic on a AMD Raven Ridge gpu. Changes
to future Mesa Vulkan drivers beyond 20.0.8 may break (or already have
broken?) page flipping on those as well.

The reason is that the classic pageflip ioctl doesn't allow a fb format
change during flip, and at least X uses classic pageflip ioctl and no
atomic modesetting api for flipping, as do all inspected Vulkan
drivers, e.g., anv, radv, amdvlk. Above commit assigns new fb->format
for use of (retiling) DCC on AMD gpu's for some tiling flags, which
is detected (and rejected) by the pageflip ioctl as a format change.

However, current atomic kms drivers hook up the ->page_flip() driver
function to the atomic helper function drm_atomic_helper_page_flip(),
which implements the legacy flip as an atomic commit. My understanding
is that a format change during flip via such an atomic commit is safe.

Therefore only reject the legacy pageflip ioctl if a fb format change
is requested on a kms driver which isn't DRIVER_ATOMIC.

This makes "legacy" flipping work again on Linux 4.11 with amdgpu-kms
and DisplayCore enabled.

Fixes: 816853f9dc40 ("drm/amd/display: Set new format info for converted metadata.")
Cc: Bas Nieuwenhuizen <bas at basnieuwenhuizen.nl>
Cc: Alex Deucher <alexander.deucher at amd.com>
Cc: David Airlie <airlied at redhat.com>
Cc: Ville Syrjälä <ville.syrjala at linux.intel.com>
Signed-off-by: Mario Kleiner <mario.kleiner.de at gmail.com>
---
 drivers/gpu/drm/drm_plane.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/drm_plane.c b/drivers/gpu/drm/drm_plane.c
index e6231947f987..4688360a078d 100644
--- a/drivers/gpu/drm/drm_plane.c
+++ b/drivers/gpu/drm/drm_plane.c
@@ -1163,7 +1163,11 @@ int drm_mode_page_flip_ioctl(struct drm_device *dev,
 	if (ret)
 		goto out;
 
-	if (old_fb->format != fb->format) {
+	/*
+	 * Format change during legacy pageflip only works if page flip is done
+	 * via an atomic commit, e.g., via drm_atomic_helper_page_flip() helper.
+	 */
+	if ((old_fb->format != fb->format) && !drm_drv_uses_atomic_modeset(dev)) {
 		DRM_DEBUG_KMS("Page flip is not allowed to change frame buffer format.\n");
 		ret = -EINVAL;
 		goto out;
-- 
2.25.1