[bug][vaapi][h264] The commit 7cbe08a930a132d84b4cf79953b00b074ec7a2a7 on certain video files leads to problems with VAAPI hardware decoding.

[Mikhail Gavrilov]

On Wed, Dec 7, 2022 at 7:58 PM Alex Deucher <alexdeucher at gmail.com> wrote:
>
>
> What GPU do you have and what entries do you have in
> sys/class/drm/card0/device/ip_discovery/die/0/UVD for the device?

I bisected the issue on the Radeon 6800M.

Parent commit for 7cbe08a930a132d84b4cf79953b00b074ec7a2a7 is
46dd2965bdd1c5a4f6499c73ff32e636fa8f9769.
For both commits ip_discovery is absent.
# ls /sys/class/drm/card0/device/ | grep ip
# ls /sys/class/drm/card1/device/ | grep ip

But from verbose info I see that player for
7cbe08a930a132d84b4cf79953b00b074ec7a2a7 use acceleration:
$ vlc -v Downloads/test_sample_480_2.mp4
VLC media player 3.0.18 Vetinari (revision )
[0000561f72097520] main libvlc: Running vlc with the default
interface. Use 'cvlc' to use vlc without interface.
[00007fa224001190] mp4 demux warning: elst box found
[00007fa224001190] mp4 demux warning: STTS table of 1 entries
[00007fa224001190] mp4 demux warning: CTTS table of 78 entries
[00007fa224001190] mp4 demux warning: elst box found
[00007fa224001190] mp4 demux warning: STTS table of 1 entries
[00007fa224001190] mp4 demux warning: elst old=0 new=1
[00007fa224d19010] faad decoder warning: decoded zero sample
[00007fa224001190] mp4 demux warning: elst old=0 new=1
[00007fa214007030] gl gl: Initialized libplacebo v4.208.0 (API v208)
libva info: VA-API version 1.16.0
libva error: vaGetDriverNameByIndex() failed with unknown libva error,
driver_name = (null)
[00007fa214007030] glconv_vaapi_x11 gl error: vaInitialize: unknown libva error
libva info: VA-API version 1.16.0
libva info: Trying to open /usr/lib64/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_16
libva info: va_openDriver() returns 0
[00007fa224c0b3a0] avcodec decoder: Using Mesa Gallium driver
23.0.0-devel for AMD Radeon RX 6800M (navi22, LLVM 15.0.4, DRM 3.42,
5.14.0-rc4-14-7cbe08a930a132d84b4cf79953b00b074ec7a2a7+) for hardware
decoding
[h264 @ 0x7fa224c3fa40] Using deprecated struct vaapi_context in decode.
[0000561f72174de0] pulse audio output warning: starting late (-9724 us)

And for 46dd2965bdd1c5a4f6499c73ff32e636fa8f9769 commit did not use
acceleration:
$ vlc -v Downloads/test_sample_480_2.mp4
VLC media player 3.0.18 Vetinari (revision )
[000055f61ad35520] main libvlc: Running vlc with the default
interface. Use 'cvlc' to use vlc without interface.
[00007fc7e8001190] mp4 demux warning: elst box found
[00007fc7e8001190] mp4 demux warning: STTS table of 1 entries
[00007fc7e8001190] mp4 demux warning: CTTS table of 78 entries
[00007fc7e8001190] mp4 demux warning: elst box found
[00007fc7e8001190] mp4 demux warning: STTS table of 1 entries
[00007fc7e8001190] mp4 demux warning: elst old=0 new=1
[00007fc7e8d19010] faad decoder warning: decoded zero sample
[00007fc7e8001190] mp4 demux warning: elst old=0 new=1
[00007fc7d8007030] gl gl: Initialized libplacebo v4.208.0 (API v208)
libva info: VA-API version 1.16.0
libva error: vaGetDriverNameByIndex() failed with unknown libva error,
driver_name = (null)
[00007fc7d8007030] glconv_vaapi_x11 gl error: vaInitialize: unknown libva error
libva info: VA-API version 1.16.0
libva info: Trying to open /usr/lib64/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_16
libva info: va_openDriver() returns 0
[00007fc7d40b3260] vaapi generic error: profile(7) is not supported
[00007fc7d8a089c0] gl gl: Initialized libplacebo v4.208.0 (API v208)
Failed to open VDPAU backend libvdpau_nvidia.so: cannot open shared
object file: No such file or directory
Failed to open VDPAU backend libvdpau_nvidia.so: cannot open shared
object file: No such file or directory
[00007fc7d89e4f80] gl gl: Initialized libplacebo v4.208.0 (API v208)
[000055f61ae12de0] pulse audio output warning: starting late (-13537 us)

So my bisect didn't make sense :(
Anyway can you reproduce the issue with the attached sample file and
vlc on fresh kernel (6.1-rc8)?

Thanks!

-- 
Best Regards,
Mike Gavrilov.