Hi all,
Recently I've been discussing with various people [1] [2] about amdgpu's handling of KMS planes. AMD hardware is a bit special when it comes to the cursor plane, and it's not always 100% clear how that maps with the KMS API.
Up until now we were using cursor and overlay planes in gamescope [3], but some changes in the amdgpu driver [1] makes us unable to use planes (and makes us fallback to composition). Basically, now the overlay cannot be used at all unless it covers the whole CRTC.
The root cause is the cursor plane. The cursor plane can only be enabled if there is an unscaled plane covering the whole CRTC right underneath.
So one might assume that as long as the cursor plane isn't used, the overlay plane could be enabled even if it doesn't cover the whole CRTC. Unfortunately, ChromeOS uses the atomic API to setup primary/overlay planes together with the legacy API to setup the cursor plane. This means that at any time ChromeOS can toggle the cursor plane and will expect the cursor plane to work.
For this reason, amdgpu now restricts the use of the overlay plane. However this penalizes all KMS users because of a single user-space implementation.
After discussion with Daniel Vetter [2], it seemed reasonable to add a ChromeOS quirk in the kernel which retains the current behavior, but require everybody else to use the atomic API properly, ie. not mix atomic overlay plane usage with legacy cursor plane usage. This would allow fully atomic user-space to better take advantage of overlay planes with amdgpu.
Does this seem like an accurate summary of the situation? Does this seem like a reasonable path forward?
Thanks,
Simon
[1]: https://lore.kernel.org/amd-gfx/SrcUnUUGJquVgjp9P79uV8sv6s-kMHG4wp0S3b4Nh9ks... [2]: https://oftc.irclog.whitequark.org/dri-devel/2021-08-27#1630077257-163007804...; [3]: https://github.com/Plagman/gamescope