[PATCH 2/2] amd/display: add cursor check for YUV primary plane

[Kazlauskas, Nicholas]

On 2021-02-19 11:19 a.m., Simon Ser wrote:
> The cursor plane can't be displayed if the primary plane isn't
> using an RGB format. Reject such atomic commits so that user-space
> can have a fallback instead of an invisible cursor.
> 
> In theory we could support YUV if the cursor is also YUV, but at the
> moment only ARGB8888 cursors are supported.

Patch 1 looks good, but this patch needs to be adjusted.

We can support cursor plane, but only if we have an overlay plane 
enabled that's using XRGB/ARGB.

This is what we do on Chrome OS for video playback:

Cursor Plane - ARGB8888
Overlay Plane - ARGB8888 Desktop/UI with cutout for video
Primary Plane - NV12 video

So this new check would break this usecase. It needs to check that there 
isn't an XRGB/ARGB plane at the top of the blending chain instead.

Regards,
Nicholas Kazlauskas

> 
> Signed-off-by: Simon Ser <contact at emersion.fr>
> Cc: Alex Deucher <alexander.deucher at amd.com>
> Cc: Harry Wentland <hwentlan at amd.com>
> Cc: Nicholas Kazlauskas <nicholas.kazlauskas at amd.com>
> ---
>   drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 7 +++++++
>   1 file changed, 7 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
> index 4548b779bbce..f659e6cfdfcf 100644
> --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
> +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
> @@ -9239,6 +9239,13 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state,
>   		return -EINVAL;
>   	}
>   
> +	/* In theory we could probably support YUV cursors when the primary
> +	 * plane uses a YUV format, but there's no use-case for it yet. */
> +	if (new_primary_state->fb && new_primary_state->fb->format->is_yuv) {
> +		drm_dbg_atomic(crtc->dev, "Cursor plane can't be used with YUV primary plane\n");
> +		return -EINVAL;
> +	}
> +
>   	return 0;
>   }
>   
>