[BUG][AMD][VDPAU] RX6600 GPU locks up when decoding HEVC

[Timur Kristóf]

Hi,

Please use the issue tracker on the Mesa Gitlab for discussing bug
reports. I recommend to include all information that you can in there,
we don't use this mailing list for bug reports.

Thanks & best regards,
Timur

On Thu, 2024-02-15 at 17:23 +0000, Chris Rankin wrote:
> Thanks, I have discovered that my PC needs for VLC to use hardware
> decoding when playing 4K HEVC streams, but that VLC won't support
> VA-API again until VLC 4 is released.
> 
> As an aside, one of these 4K HEVC streams correctly uses VDPAU if I
> use:
> 
> $ mpv --vo=gpu --hwdec=vdpau --hwdec-image-format=yuv420p
> 
> but uses software decoding if I remove the
> '--hwdec-image-format=yuv420p' parameter because:
> 
> [vd] Opening decoder hevc
> [vd] Looking at hwdec hevc-vdpau...
> [vd] Trying hardware decoding via hevc-vdpau.
> [vd] Selected codec: hevc (HEVC (High Efficiency Video Coding))
> [vd] Pixel formats supported by decoder: vaapi vdpau cuda yuv420p10le
> [vd] Codec profile: Main 10 (0x2)
> [ffmpeg] AVHWFramesContext: Unsupported sw format: yuv420p10le
> Failed to allocate hw frames.
> [vd] Requesting pixfmt 'yuv420p10le' from decoder.
> [vd] Attempting next decoding method after failure of hevc-vdpau.
> [vd] Skipping previously attempted hwdec: hevc-vdpau
> [vd] Using software decoding.
> 
> Does this look like another Mesa / VDPAU bug (i.e. should I raise a
> new issue for it?), or is it actually a problem with mpv?
> 
> Cheers,
> Chris
> 
> On Thu, 15 Feb 2024 at 16:01, Liu, Leo <Leo.Liu at amd.com> wrote:
> > 
> > [AMD Official Use Only - General]
> > 
> > This is being looked.
> > 
> > > -----Original Message-----
> > > From: mesa-dev <mesa-dev-bounces at lists.freedesktop.org> On Behalf
> > > Of
> > > Chris Rankin
> > > Sent: Thursday, February 15, 2024 10:38 AM
> > > To: mesa-dev at lists.freedesktop.org
> > > Subject: [BUG][AMD][VDPAU] RX6600 GPU locks up when decoding HEVC
> > > 
> > > I have discovered that my RX6600 cannot use VDPAU to decode HEVC,
> > > although
> > > VA-API works OK. Unfortunately, VLC 3.x only supports VDPAU these
> > > days for
> > > hardware decoding.
> > > 
> > > I have raised this issue:
> > > https://gitlab.freedesktop.org/mesa/mesa/-/issues/10599
> > > 
> > > Mesa 23.3.5 just used to fail, but both Mesa 24.0.1 and 24.1.0-
> > > devel actually
> > > lock up the GPU and force me to reboot my PC.
> > > 
> > > According to vdpauinfo:
> > > 
> > > name                        level macbs width height
> > > ----------------------------------------------------
> > > ...
> > > HEVC_MAIN                      186 139264  8192  4352
> > > HEVC_MAIN_10                   186 139264  8192  4352
> > > HEVC_MAIN_STILL                --- not supported ---
> > > HEVC_MAIN_12                   --- not supported ---
> > > HEVC_MAIN_444                  --- not supported ---
> > > HEVC_MAIN_444_10               --- not supported ---
> > > HEVC_MAIN_444_12               --- not supported ---
> > > ...
> > > 
> > > Cheers,
> > > Chris