amdgpu: atomic API and cursor/overlay planes

[Harry Wentland]

On 2021-09-07 10:03, Simon Ser wrote:
> Hi all,
> 
> Recently I've been discussing with various people [1] [2] about amdgpu's
> handling of KMS planes. AMD hardware is a bit special when it comes to
> the cursor plane, and it's not always 100% clear how that maps with the
> KMS API.
> 
> Up until now we were using cursor and overlay planes in gamescope [3],
> but some changes in the amdgpu driver [1] makes us unable to use planes
> (and makes us fallback to composition). Basically, now the overlay
> cannot be used at all unless it covers the whole CRTC.
> 
> The root cause is the cursor plane. The cursor plane can only be
> enabled if there is an unscaled plane covering the whole CRTC right
> underneath.
> 

A bit of background on why that is:

DRM thinks of cursor and plane blending like this:

┌─────────┐
│ Cursor  ├───────┐
└─────────┘       │
                  │
┌─────────┐      ┌▼──────┐
│ Plane 1 ├──────► CRTC  │
└─────────┘      └▲──────┘
                  │
┌─────────┐       │
│ Plane 2 ├───────┘
└─────────┘

AMD HW is designed like this:


┌─────────┐   ┌─────────┐      ┌───────┐
│ Cursor  ├───► Plane 1 ├──────► CRTC  │
└─────────┘   └─────────┘      └▲──────┘
                                │
              ┌─────────┐       │
              │ Plane 2 ├───────┘
              └─────────┘

It means that the cursor cannot be drawn outside it's associated
plane, is scaled with the plane, and treated as being in the same
color space as the plane.

> So one might assume that as long as the cursor plane isn't used, the
> overlay plane could be enabled even if it doesn't cover the whole CRTC.
> Unfortunately, ChromeOS uses the atomic API to setup primary/overlay
> planes together with the legacy API to setup the cursor plane. This
> means that at any time ChromeOS can toggle the cursor plane and will
> expect the cursor plane to work.
> 
> For this reason, amdgpu now restricts the use of the overlay plane.
> However this penalizes all KMS users because of a single user-space
> implementation.
> 
> After discussion with Daniel Vetter [2], it seemed reasonable to add
> a ChromeOS quirk in the kernel which retains the current behavior, but
> require everybody else to use the atomic API properly, ie. not mix
> atomic overlay plane usage with legacy cursor plane usage. This would
> allow fully atomic user-space to better take advantage of overlay
> planes with amdgpu.
> 
> Does this seem like an accurate summary of the situation? Does this
> seem like a reasonable path forward?
> 

I think it's a good summary.

Won't adding a ChromeOS quirk break old ChromeOS userspace with a new
kernel?

Though it seems to me that the ChromeOS compositor is doing atomic
wrong when it comes to the cursor, so the proposed solution might be
a decent temporary path forward until ChromeOS teaches their compositor
to not crash when cursor enablement causes atomic_check to fail.

Sean, any thoughts?

Thanks,
Harry

> Thanks,
> 
> Simon
> 
> [1]: https://lore.kernel.org/amd-gfx/SrcUnUUGJquVgjp9P79uV8sv6s-kMHG4wp0S3b4Nh9ksi29EIOye5edofuXkDLRvGfvkkRpQZ9JM7MNqew2B3kFUhaxsonDRXprkAYXaQUo=@emersion.fr/>> [2]: https://oftc.irclog.whitequark.org/dri-devel/2021-08-27#1630077257-1630078046>> [3]: https://github.com/Plagman/gamescope>>