[PATCH v3 3/4] amd/display: add cursor rotation check

[Harry Wentland]

On 2023-08-17 09:44, Harry Wentland wrote:
> On 2023-08-17 06:53, Simon Ser wrote:
>> The commit 1347385fe187 ("drm/amd/display: don't expose rotation
>> prop for cursor plane") removed the rotation property for the
>> cursor plane, assuming the cursor would always be displayed without
>> any rotation. However the rotation is inherited from the underlying
>> plane.
>>
>> As a result, if the primary plane is rotated, then the cursor plane
>> will incorrectly be rotated as well even though it doesn't have a
>> rotation property.
>>
>> To fix this, check that the underlying plane isn't rotated.
>>
> 
> HW cursor works with a rotated screen on KDE on AMD HW. This
> looks like it will break that.
> 

Sorry, scrap that. KDE seems to do a SW rotation with HW cursor.

Harry

> Harry
> 
>> Signed-off-by: Simon Ser <contact at emersion.fr>
>> Cc: Alex Deucher <alexander.deucher at amd.com>
>> Cc: Harry Wentland <hwentlan at amd.com>
>> Cc: Nicholas Kazlauskas <nicholas.kazlauskas at amd.com>
>> Cc: Michel Dänzer <michel at daenzer.net>
>> ---
>>  drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 7 +++++++
>>  1 file changed, 7 insertions(+)
>>
>> diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
>> index df40f74f5a15..aba01255ba12 100644
>> --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
>> +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
>> @@ -9867,6 +9867,13 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state,
>>  			return -EINVAL;
>>  		}
>>  
>> +		if (new_underlying_state->rotation != DRM_MODE_ROTATE_0) {
>> +			drm_dbg_atomic(crtc->dev,
>> +				       "Cursor [PLANE:%d:%s] can't be used with rotated underlying [PLANE:%d:%s]\n",
>> +				       cursor->base.id, cursor->name, underlying->base.id, underlying->name);
>> +			return -EINVAL;
>> +		}
>> +
>>  		/* If this plane covers the whole CRTC, no need to check planes underneath */
>>  		if (new_underlying_state->crtc_x <= 0 &&
>>  		    new_underlying_state->crtc_y <= 0 &&
>