6.11/regression/bisected - after commit 1b04dcca4fb1, launching some RenPy games causes computer hang

[Leo Li]

Hi Mikhail,

Can you give this patch a try to see if it helps?
https://gist.github.com/leeonadoh/3271e90ec95d768424c572c970ada743

Thanks,
Leo

On 2024-09-10 11:47, Leo Li wrote:
> 
> 
> On 2024-09-08 19:30, Mikhail Gavrilov wrote:
>> I have done additional tests:
>> 1. The computer does not hang with 6900XT instead the screen flickers
>> when moving the cursor.
>> 2. The computer does not hang with 7900XTX if I turn off VRR. But the
>> screen flickers when moving the cursor, as on 6900XT.
>> To enable VRR, please set 'variable-refresh-rate' in
>> experimental-features, and in the Display setting, enable Variable
>> Refresh Rate.
>> $ gsettings set org.gnome.mutter experimental-features
>> "['variable-refresh-rate', 'scale-monitor-framebuffer']"
>> https://postimg.cc/PvXYdvGR
> 
> Thanks Mikhail, I think I know what's going on now.
> 
> The `scale-monitor-framebuffer` experimental setting is what puts us down the
> bad code path. It seems VRR has nothing to do with this issue, just setting
> `scale-monitor-framebuffer` is enough to reproduce.
> 
> It seems that mutter with this setting is opting for HW scaling rather than GPU
> scaling. I see that "Find the Orange Narwhal" sends out a 1080p buffer,
> which with this setting, gets directly scanned out and scaled by DCN HW to 4k in
> full screen.
> 
> An oddity with current gen DCN hardware is that the cursor inherits the scaling
> of the HW plane underneath. So if mutter requests a hw cursor with a different
> scaling than the game's plane, amdgpu will reject that, and likely force mutter
> into SW cursor.
> 
> My offending patch changed this behavior by rerouting DCN HW pipes to
> accommodate such a configuration. It essentially takes a full-fledged DCN
> overlay plane, and uses that just for the cursor, and thereby freeing it from
> inheriting things from the underlying hw plane.
> 
> My guess is this causes flickering due to how DC (display core driver) handles
> updates; it needs all enabled planes in it's update state. However, a KMS cursor
> update will only include the cursor plane. It's likely that amdgpu_dm only adds
> the dedicated cursor plane to DC's update state, leaving the game's plane out.
> 
> The fix isn't exactly trivial. If I don't get anywhere before the fixes window,
> I'll send out a revert.
> 
> Cheers,
> Leo