[BUG][AMD][VDPAU] RX6600 GPU locks up when decoding HEVC

[Chris Rankin]

Hi,

I have discovered that my RX6600 cannot use VDPAU to decode HEVC,
although VA-API works OK. Unfortunately, VLC 3.x only supports VDPAU
these days for hardware decoding.

I have raised this issue:
https://gitlab.freedesktop.org/mesa/mesa/-/issues/10599

Mesa 23.3.5 just used to fail, but both Mesa 24.0.1 and 24.1.0-devel
actually lock up the GPU and force me to reboot my PC.

According to vdpauinfo:

name                        level macbs width height
----------------------------------------------------
...
HEVC_MAIN                      186 139264  8192  4352
HEVC_MAIN_10                   186 139264  8192  4352
HEVC_MAIN_STILL                --- not supported ---
HEVC_MAIN_12                   --- not supported ---
HEVC_MAIN_444                  --- not supported ---
HEVC_MAIN_444_10               --- not supported ---
HEVC_MAIN_444_12               --- not supported ---
...

Cheers,
Chris