6.11/regression/bisected - after commit 1b04dcca4fb1, launching some RenPy games causes computer hang

[Mikhail Gavrilov]

Hi,
After commit 1b04dcca4fb1, launching some RenPy games causes computer hang.
After the hang, even Alt + sysrq + REISUB can't reboot the computer!
And no trace in the kernel log!
For demonstration, I'm going to use the game "Find the Orange Narwhal"
because it is free and has 100% reproducivity for this issue.
You can find it in the Steam Store:
https://store.steampowered.com/app/2946010/Find_the_Orange_Narwhal/
I uploaded demonstration video to youtube: https://youtu.be/yVW6rImRpXw

Unfortunately, I can't check the revert commit 1541d63c5fe2 because of
conflicts.

mikhail at primary-ws ~/p/g/linux (master)> git reset v6.11-rc1 --hard
HEAD is now at 8400291e289e Linux 6.11-rc1

mikhail at primary-ws ~/p/g/linux (master)> git revert -n 1b04dcca4fb1
Auto-merging drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
CONFLICT (content): Merge conflict in
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
Auto-merging drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h
Auto-merging drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c
Auto-merging drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
CONFLICT (content): Merge conflict in
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
error: could not revert 1b04dcca4fb1... drm/amd/display: Introduce
overlay cursor mode
hint: after resolving the conflicts, mark the corrected paths
hint: with 'git add <paths>' or 'git rm <paths>'
hint: Disable this message with "git config advice.mergeConflict false"

commit 1b04dcca4fb10dd3834893a60de74edd99f2bfaf
Author: Leo Li <sunpeng.li at amd.com>
Date:   Thu Jan 18 16:29:49 2024 -0500

    drm/amd/display: Introduce overlay cursor mode

    [Why]

    DCN is the display hardware for amdgpu. DRM planes are backed by DCN
    hardware pipes, which carry pixel data from one end (memory), to the
    other (output encoder).

    Each DCN pipe has the ability to blend in a cursor early on in the
    pipeline. In other words, there are no dedicated cursor planes in DCN,
    which makes cursor behavior somewhat unintuitive for compositors.

    For example, if the cursor is in RGB format, but the top-most DRM plane
    is in YUV format, DCN will not be able to blend them. Because of this,
    amdgpu_dm rejects all configurations where a cursor needs to be enabled
    on top of a YUV formatted plane.

    From a compositor's perspective, when computing an allocation for
    hardware plane offloading, this cursor-on-yuv configuration result in an
    atomic test failure. Since the failure reason is not obvious at all,
    compositors will likely fall back to full rendering, which is not ideal.

    Instead, amdgpu_dm can try to accommodate the cursor-on-yuv
    configuration by opportunistically reserving a separate DCN pipe just
    for the cursor. We can refer to this as "overlay cursor mode". It is
    contrasted with "native cursor mode", where the native DCN per-pipe
    cursor is used.

    [How]

    On each crtc, compute whether the cursor plane should be enabled in
    overlay mode. If it is, mark the CRTC as requesting overlay cursor mode.

    Overlay cursor should be enabled whenever there exists a underlying
    plane that has YUV format, or is scaled differently than the cursor. It
    should also be enabled if there is no underlying plane, or if underlying
    planes do not cover the entire CRTC.

    During DC validation, attempt to enable a separate DCN pipe for the
    cursor if it's in overlay mode. If that fails, or if no overlay mode is
    requested, then fallback to native mode.

    v2:
    * Update commit message for when overlay cursor should be enabled
    * Also consider scale and no-underlying-plane case (cursor on crtc bg)
    * Consider all underlying planes when determinig overlay/native, not
      just the plane immediately beneath the cursor, as it may not cover the
      entire CRTC.
    * Fix typo s/decending/descending/
    * Force native cursor on pre-DCN hardware

    Reviewed-by: Harry Wentland <harry.wentland at amd.com>
    Acked-by: Zaeem Mohamed <zaeem.mohamed at amd.com>
    Signed-off-by: Leo Li <sunpeng.li at amd.com>
    Acked-by: Harry Wentland <harry.wentland at amd.com>
    Acked-by: Pekka Paalanen <pekka.paalanen at collabora.com>
    Tested-by: Daniel Wheeler <daniel.wheeler at amd.com>
    Signed-off-by: Alex Deucher <alexander.deucher at amd.com>

 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c       | 490
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------------------------------------------
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h       |   7 +++
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c  |   1 +
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c |  13 ++++-
 4 files changed, 389 insertions(+), 122 deletions(-)


My hardware specs are: https://linux-hardware.org/?probe=61bd7390a9

Leo, can you look into it, please?

-- 
Best Regards,
Mike Gavrilov.