[PATCH v3 3/4] amd/display: add cursor rotation check

[Simon Ser]

The commit 1347385fe187 ("drm/amd/display: don't expose rotation
prop for cursor plane") removed the rotation property for the
cursor plane, assuming the cursor would always be displayed without
any rotation. However the rotation is inherited from the underlying
plane.

As a result, if the primary plane is rotated, then the cursor plane
will incorrectly be rotated as well even though it doesn't have a
rotation property.

To fix this, check that the underlying plane isn't rotated.

Signed-off-by: Simon Ser <contact at emersion.fr>
Cc: Alex Deucher <alexander.deucher at amd.com>
Cc: Harry Wentland <hwentlan at amd.com>
Cc: Nicholas Kazlauskas <nicholas.kazlauskas at amd.com>
Cc: Michel Dänzer <michel at daenzer.net>
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
index df40f74f5a15..aba01255ba12 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
@@ -9867,6 +9867,13 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state,
 			return -EINVAL;
 		}
 
+		if (new_underlying_state->rotation != DRM_MODE_ROTATE_0) {
+			drm_dbg_atomic(crtc->dev,
+				       "Cursor [PLANE:%d:%s] can't be used with rotated underlying [PLANE:%d:%s]\n",
+				       cursor->base.id, cursor->name, underlying->base.id, underlying->name);
+			return -EINVAL;
+		}
+
 		/* If this plane covers the whole CRTC, no need to check planes underneath */
 		if (new_underlying_state->crtc_x <= 0 &&
 		    new_underlying_state->crtc_y <= 0 &&
-- 
2.41.0