[PATCH v3 3/4] amd/display: add cursor rotation check

[Harry Wentland]

On 2023-08-17 06:53, Simon Ser wrote:
> The commit 1347385fe187 ("drm/amd/display: don't expose rotation
> prop for cursor plane") removed the rotation property for the
> cursor plane, assuming the cursor would always be displayed without
> any rotation. However the rotation is inherited from the underlying
> plane.
> 
> As a result, if the primary plane is rotated, then the cursor plane
> will incorrectly be rotated as well even though it doesn't have a
> rotation property.
> 
> To fix this, check that the underlying plane isn't rotated.
> 

HW cursor works with a rotated screen on KDE on AMD HW. This
looks like it will break that.

Harry

> Signed-off-by: Simon Ser <contact at emersion.fr>
> Cc: Alex Deucher <alexander.deucher at amd.com>
> Cc: Harry Wentland <hwentlan at amd.com>
> Cc: Nicholas Kazlauskas <nicholas.kazlauskas at amd.com>
> Cc: Michel Dänzer <michel at daenzer.net>
> ---
>  drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
> index df40f74f5a15..aba01255ba12 100644
> --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
> +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
> @@ -9867,6 +9867,13 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state,
>  			return -EINVAL;
>  		}
>  
> +		if (new_underlying_state->rotation != DRM_MODE_ROTATE_0) {
> +			drm_dbg_atomic(crtc->dev,
> +				       "Cursor [PLANE:%d:%s] can't be used with rotated underlying [PLANE:%d:%s]\n",
> +				       cursor->base.id, cursor->name, underlying->base.id, underlying->name);
> +			return -EINVAL;
> +		}
> +
>  		/* If this plane covers the whole CRTC, no need to check planes underneath */
>  		if (new_underlying_state->crtc_x <= 0 &&
>  		    new_underlying_state->crtc_y <= 0 &&