[PATCH v3 1/4] amd/display: add cursor check for YUV underlying pipe

[Simon Ser]

The cursor plane can't be displayed if the underlying pipe isn't
using an RGB format. Reject such atomic commits so that user-space
can have a fallback instead of an invisible cursor.

In theory we could support YUV if the cursor is also YUV, but at the
moment only ARGB8888 cursors are supported.

Signed-off-by: Simon Ser <contact at emersion.fr>
Cc: Alex Deucher <alexander.deucher at amd.com>
Cc: Harry Wentland <hwentlan at amd.com>
Cc: Nicholas Kazlauskas <nicholas.kazlauskas at amd.com>
Cc: Michel Dänzer <michel at daenzer.net>
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
index 268cb99a4c4b..f60858112d3d 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
@@ -9844,6 +9844,15 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state,
 			return -EINVAL;
 		}
 
+		/* In theory we could probably support YUV cursors when the underlying
+		 * plane uses a YUV format, but there's no use-case for it yet. */
+		if (new_underlying_state->fb->format->is_yuv) {
+			drm_dbg_atomic(crtc->dev,
+				       "Cursor [PLANE:%d:%s] can't be used with YUV underlying [PLANE:%d:%s]\n",
+				       cursor->base.id, cursor->name, underlying->base.id, underlying->name);
+			return -EINVAL;
+		}
+
 		/* If this plane covers the whole CRTC, no need to check planes underneath */
 		if (new_underlying_state->crtc_x <= 0 &&
 		    new_underlying_state->crtc_y <= 0 &&
-- 
2.41.0