[Mesa-dev] VAAPI encoder and OpenGL
Daniel Gomez
daniel at qtec.com
Tue Aug 11 16:05:24 UTC 2020
Hi Thong,
Thanks for the support.
I think the problem is on the mesa vaapi driver (libva-vaapi-driver) somehow
but I tried your firmware anyway and the problem persists in regards to the
encoder.
Now, I have updated my system and I'm running the following:
root at qt5122:/mnt# vainfo
error: XDG_RUNTIME_DIR not set in the environment.
error: can't connect to X server!
libva info: VA-API version 1.8.0
libva info: Trying to open /usr/lib/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_8
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.8 (libva 2.8.0)
vainfo: Driver version: Mesa Gallium driver 20.0.2 for AMD Radeon R7
Graphics (CARRIZO, DRM 3.36.0, 5.5.0-qtec-standard, LLVM 8.0.0)
vainfo: Supported profile and entrypoints
VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointVLD
VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264High : VAEntrypointVLD
VAProfileH264High : VAEntrypointEncSlice
VAProfileHEVCMain : VAEntrypointVLD
VAProfileJPEGBaseline : VAEntrypointVLD
VAProfileNone : VAEntrypointVideoProc
As you can see, I'm using mesa 20.0.2 and libva 2.8.0.
Running the following gstreamer pipelines I can see how the vaapi driver fails:
gstreamer pipelines:
Gst.parse_launch(
"udpsrc address=224.1.1.1 port=5000
multicast-iface=enx24f5a2f1ba7a ! " \
"application/x-rtp,media=video,payload=33,clock-rate=90000,encoding-name=MP2T
! " \
"rtpmp2tdepay ! tsdemux name=td td.video_0_0041 ! multiqueue !
h264parse ! " \
"queue ! vaapidecodebin !
video/x-raw,format=NV12,width=1920,height=1080 ! queue ! " \
"vaapih264enc ! queue ! fakesink dump=1"
)
Gst.parse_launch(
"filesrc name=fsrc location=/mnt/tmp/theoffice.ts ! " \
"tsdemux name=td td.video_0_0041 ! h264parse ! " \
"queue name=qvaapidec2in ! vaapidecodebin ! " \
"video/x-raw,format=NV12,width=1920,height=1080 ! " \
"queue name=qvaapidec2out ! intervideosink channel=chv1"
)
pipeline2 = Gst.parse_launch(
"intervideosrc channel=chv1 ! videoconvert ! " \
"video/x-raw,format=NV12,width=1920,height=1080 ! " \
"queue name=qvaapienc2in ! vaapih264enc name=enc " \
"rate-control=4 bitrate=25000 ! video/x-h264,profile=high ! " \
"queue name=qvaapienc2out ! " \
"fakesink dump={}".format(encdump)
)
Note: I can send the script if you want so, it's easier to replicate it.
GStreamer log:
0:00:00.464680040 948 0x55f0ec6b4b70 ERROR vaapivideomemory
gstvaapivideomemory.c:741:gst_video_info_update_from_surface: Cannot
create a VA derived image from surface 0x55f0ec6de4a0
0:00:00.502405919 948 0x55f0ec6b4f20 FIXME default
gstutils.c:3981:gst_pad_create_stream_id_internal:<intervideosrc0:src>
Creating random stream-id, consider implementing a deterministic way
of creating a stream-id
0:00:00.528549645 948 0x55f0ec6b4d90 ERROR vaapivideomemory
gstvaapivideomemory.c:741:gst_video_info_update_from_surface: Cannot
create a VA derived image from surface 0x55f0ec6e0b20
0:00:00.529446550 948 0x55f0ec6b4d90 TRACE GST_TRACER
:0:: queuelevel, queue=(string)qvaapienc2in, size_bytes=(uint)0,
max_size_bytes=(uint)10485760, size_buffers=(uint)0,
max_size_buffers=(uint)200, size_time=(string)0:00:00.000000000,
max_size_time=(string)0:00:01.000000000;
0:00:00.529557834 948 0x55f0ec6b4d90 ERROR vaapi
gstvaapiencoder.c:543:gst_vaapi_encoder_put_frame: failed to encode
frame (status = -1)
0:00:00.529575486 948 0x55f0ec6b4d90 ERROR vaapiencode
gstvaapiencode.c:722:gst_vaapiencode_handle_frame: failed to encode
frame 0 (status -1)
So, I guess that's the reason I was seeing at the beginning how the queue
previous to the encoder was filled up to the maximum and therefore,
buffers were never consumed by the vaapi encoder.
At the same time, I tried to replicate the same environment in my laptop
by using the VAAPI driver provided by intel (intel-vaapi-driver) instead of
the one provided by mesa (libva-mesa-driver) with the same gstreamer-vaapi,
gstreamer and libva versions and the problem only occurs on the setup where
I'm running the mesa vaapi driver.
Any ideas?
Thanks
On Thu, 6 Aug 2020 at 23:42, Thai, Thong <Thong.Thai at amd.com> wrote:
>
> [AMD Official Use Only - Internal Distribution Only]
>
>
> [AMD Official Use Only - Internal Distribution Only]
>
>
>
> Hi Daniel,
>
>
>
> Can you try the attached firmware file and see if it works, and/or fixes your issue?
>
>
>
> Save it to `/lib/firmware/amdgpu/` or `/lib/firmware/$(uname -r)/amdgpu` (if you’ve installed the amdgpu driver package) and run `update-initramfs -c -k $(uname -r)` if you’re using Ubuntu, to update your initramfs.
>
>
>
> Regards,
>
>
>
> Thong Thai
>
>
>
> From: Thai, Thong <Thong.Thai at amd.com>
> Sent: Thursday, August 6, 2020 2:41 PM
> To: Daniel Gomez <daniel at qtec.com>; Liu, Leo <Leo.Liu at amd.com>
> Cc: mesa-dev at lists.freedesktop.org; Zhang, Boyuan <Boyuan.Zhang at amd.com>
> Subject: Re: [Mesa-dev] VAAPI encoder and OpenGL
>
>
>
> Hi Daniel,
>
>
>
> Unfortunately, I wasn't able to recreate the issue on a Polaris card, so the issue might only be with older hardware. I'll see if there's any updated firmware for the Carizzo devices.
>
>
>
> Regards,
>
> Thong Thai
>
> ________________________________
>
> From: Daniel Gomez <daniel at qtec.com>
> Sent: August 4, 2020 9:48 AM
> To: Liu, Leo <Leo.Liu at amd.com>
> Cc: mesa-dev at lists.freedesktop.org <mesa-dev at lists.freedesktop.org>; Zhang, Boyuan <Boyuan.Zhang at amd.com>; Thai, Thong <Thong.Thai at amd.com>
> Subject: Re: [Mesa-dev] VAAPI encoder and OpenGL
>
>
>
> Hi guys,
>
> any update on this?
>
> Thanks
>
>
>
> On Mon, 27 Apr 2020 at 16:23, Daniel Gomez <daniel at qtec.com> wrote:
> >
> > Hi,
> >
> > We have also try to use VAAPI decoder + OpenGL with mpv with the same effects:
> >
> > VAAPI + OpenGL example:
> > DISPLAY=:0 mpv --hwdec=vaapi --vo=opengl The\ Simpsons\ Movie\ -\
> > 1080p\ Trailer.mp4
> > Playing: The Simpsons Movie - 1080p Trailer.mp4
> > [osd/libass] Error opening memory font 'fonts.conf'
> > [ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: stream 0, timescale not set
> > (+) Video --vid=1 (*) (h264 1920x800 23.976fps)
> > Video --vid=2 [P] (png)
> > (+) Audio --aid=1 --alang=und (*) (aac 2ch 44100Hz)
> > File tags:
> > Artist: 20th Century Fox
> > Genre: Trailer
> > Title: The Simpsons Movie - Trailer
> > [vo/opengl/x11] XOpenIM() failed. Unicode input will not work.
> > [vo/opengl/x11] XOpenIM() failed. Unicode input will not work.
> > [vo/opengl/vaapi-egl] vaDeriveImage(): invalid VASurfaceID
> > [vo/opengl/vaapi-egl] vaDeriveImage(): invalid VAImageFormat
> > [ffmpeg] AVHWFramesContext: Failed to create surface: 2 (resource
> > allocation failed).
> > [ffmpeg] AVHWFramesContext: Unable to allocate a surface from internal
> > buffer pool.
> > [vo/opengl/vaapi-egl] vaDeriveImage(): invalid VASurfaceID
> > [vo/opengl/vaapi-egl] vaDeriveImage(): invalid VAImageFormat
> > [ffmpeg] AVHWFramesContext: Failed to create surface: 2 (resource
> > allocation failed).
> > [ffmpeg] AVHWFramesContext: Unable to allocate a surface from internal
> > buffer pool.
> > VO does not support requested hardware decoder, or loading it failed.
> > ALSA lib ../../../alsa-lib-1.1.8/src/pcm/pcm_dmix.c:1108:(snd_pcm_dmix_open)
> > unable to open slave
> > [ao/alsa] Playback open error: No such file or directory
> > [ao/oss] Can't open audio device /dev/dsp: No such file or directory
> > [ao] Failed to initialize audio driver 'oss'
> > Could not open/initialize audio device -> no sound.
> > Audio: no audio
> > VO: [opengl] 1920x800 yuv420p
> > V: 00:00:06 / 00:02:17 (4%)
> >
> > Just opengl or vappi works fine.
> >
> > VAAPI example:
> > DISPLAY=:0 mpv --hwdec=vaapi --vo=xv The\ Simpsons\ Movie\ -\ 1080p\
> > Trailer.mp4
> > Playing: The Simpsons Movie - 1080p Trailer.mp4
> > [osd/libass] Error opening memory font 'fonts.conf'
> > [ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: stream 0, timescale not set
> > (+) Video --vid=1 (*) (h264 1920x800 23.976fps)
> > Video --vid=2 [P] (png)
> > (+) Audio --aid=1 --alang=und (*) (aac 2ch 44100Hz)
> > File tags:
> > Artist: 20th Century Fox
> > Genre: Trailer
> > Title: The Simpsons Movie - Trailer
> > [vo/xv/x11] XOpenIM() failed. Unicode input will not work.
> > [vo/xv] Warning: this legacy VO has bad quality and performance, and
> > will in particular result in blurry OSD and subtitles. You should fix
> > your graphics drivers, or not force the xv VO.
> > VO does not support requested hardware decoder, or loading it failed.
> > ALSA lib ../../../alsa-lib-1.1.8/src/pcm/pcm_dmix.c:1108:(snd_pcm_dmix_open)
> > unable to open slave
> > [ao/alsa] Playback open error: No such file or directory
> > [ao/oss] Can't open audio device /dev/dsp: No such file or directory
> > [ao] Failed to initialize audio driver 'oss'
> > Could not open/initialize audio device -> no sound.
> > Audio: no audio
> > VO: [xv] 1920x800 yuv420p
> > V: 00:00:09 / 00:02:17 (6%)
> > Exiting... (Quit)
> >
> > OpenGL example:
> > DISPLAY=:0 mpv --hwdec=no --vo=opengl The\ Simpsons\ Movie\ -\ 1080p\
> > Trailer.mp4
> > Playing: The Simpsons Movie - 1080p Trailer.mp4
> > [osd/libass] Error opening memory font 'fonts.conf'
> > [ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: stream 0, timescale not set
> > (+) Video --vid=1 (*) (h264 1920x800 23.976fps)
> > Video --vid=2 [P] (png)
> > (+) Audio --aid=1 --alang=und (*) (aac 2ch 44100Hz)
> > File tags:
> > Artist: 20th Century Fox
> > Genre: Trailer
> > Title: The Simpsons Movie - Trailer
> > [vo/opengl/x11] XOpenIM() failed. Unicode input will not work.
> > [vo/opengl/x11] XOpenIM() failed. Unicode input will not work.
> > ALSA lib ../../../alsa-lib-1.1.8/src/pcm/pcm_dmix.c:1108:(snd_pcm_dmix_open)
> > unable to open slave
> > [ao/alsa] Playback open error: No such file or directory
> > [ao/oss] Can't open audio device /dev/dsp: No such file or directory
> > [ao] Failed to initialize audio driver 'oss'
> > Could not open/initialize audio device -> no sound.
> > Audio: no audio
> > VO: [opengl] 1920x800 yuv420p
> > V: 00:00:01 / 00:02:17 (1%)
> > Exiting... (Quit)
> >
> > On Mon, 27 Apr 2020 at 15:13, Leo Liu <leo.liu at amd.com> wrote:
> > >
> > > +Thong.
> > >
> > > On 2020-04-27 8:29 a.m., Daniel Gomez wrote:
> > > > Adding Boyuan Zhang to the thread.
> > > >
> > > > On Mon, 27 Apr 2020 at 14:00, Daniel Gomez <daniel at qtec.com> wrote:
> > > >> Hi guys,
> > > >>
> > > >> We have found a limitation on the usage of the GPU and the encoder
> > > >> when you run it simultaneously in a gstreamer pipeline and we would like to
> > > >> report it and try to find a solution.
> > > >>
> > > >> Besides encoding the video frame, the following gstreamer pipeline shows the
> > > >> usage of the GPU in two ways:
> > > >> 1. OpenGL color conversion.
> > > >> 2. Render to the display using OpenGL.
> > > >>
> > > >> DISPLAY=:0 \
> > > >> GST_DEBUG="*:3,GST_TRACER:7" GST_TRACERS="queuelevel" \
> > > >> gst-launch-1.0 videotestsrc ! video/x-raw,format=RGBA ! \
> > > >> glupload ! glcolorconvert ! gldownload ! \
> > > >> queue max-size-bytes=279936000 max-size-time=3000000000 leaky=2 ! \
> > > >> vaapih264enc ! \
> > > >> fakesink \
> > > >> videotestsrc ! video/x-raw ! glimagesink sync=false
> > > >>
> > > >> By running the above pipeline you can see how the encoder queue fills up to the
> > > >> maximum and never consumes the incoming data.
> > > >>
> > > >> Hardware information:
> > > >>
> > > >> AMD Embedded R-Series RX-421BD Radeon R7
> > > >>
> > > >> lspci -v -s 00:01.0
> > > >> 00:01.0 VGA compatible controller: Advanced Micro Devices, Inc.
> > > >> [AMD/ATI] Wani [Radeon R5/R6/R7 Graphics] (rev 84) (prog-if 00 [VGA
> > > >> controller])
> > > >> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Wani [Radeon
> > > >> R5/R6/R7 Graphics]
> > > >> Flags: bus master, fast devsel, latency 0, IRQ 53
> > > >> Memory at e0000000 (64-bit, prefetchable) [size=256M]
> > > >> Memory at f2000000 (64-bit, prefetchable) [size=8M]
> > > >> I/O ports at 5000 [size=256]
> > > >> Memory at f2c00000 (32-bit, non-prefetchable) [size=256K]
> > > >> Expansion ROM at 000c0000 [disabled] [size=128K]
> > > >> Capabilities: [48] Vendor Specific Information: Len=08 <?>
> > > >> Capabilities: [50] Power Management version 3
> > > >> Capabilities: [58] Express Root Complex Integrated Endpoint, MSI 00
> > > >> Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
> > > >> Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
> > > >> Capabilities: [270] Secondary PCI Express <?>
> > > >> Capabilities: [2b0] Address Translation Service (ATS)
> > > >> Capabilities: [2c0] Page Request Interface (PRI)
> > > >> Capabilities: [2d0] Process Address Space ID (PASID)
> > > >> Kernel driver in use: amdgpu
> > > >> Kernel modules: amdgpu
> > > >>
> > > >> DISPLAY=:0 glxinfo -B
> > > >> name of display: :0
> > > >> display: :0 screen: 0
> > > >> direct rendering: Yes
> > > >> Extended renderer info (GLX_MESA_query_renderer):
> > > >> Vendor: X.Org (0x1002)
> > > >> Device: AMD Radeon R7 Graphics (CARRIZO, DRM 3.36.0,
> > > >> 5.5.0-qtec-standard, LLVM 8.0.0) (0x9874)
> > > >> Version: 19.0.1
> > > >> Accelerated: yes
> > > >> Video memory: 1024MB
> > > >> Unified memory: no
> > > >> Preferred profile: core (0x1)
> > > >> Max core profile version: 4.5
> > > >> Max compat profile version: 4.5
> > > >> Max GLES1 profile version: 1.1
> > > >> Max GLES[23] profile version: 3.2
> > > >> OpenGL vendor string: X.Org
> > > >> OpenGL renderer string: AMD Radeon R7 Graphics (CARRIZO, DRM 3.36.0,
> > > >> 5.5.0-qtec-standard, LLVM 8.0.0)
> > > >> OpenGL core profile version string: 4.5 (Core Profile) Mesa 19.0.1
> > > >> OpenGL core profile shading language version string: 4.50
> > > >> OpenGL core profile context flags: (none)
> > > >> OpenGL core profile profile mask: core profile
> > > >>
> > > >> OpenGL version string: 4.5 (Compatibility Profile) Mesa 19.0.1
> > > >> OpenGL shading language version string: 4.50
> > > >> OpenGL context flags: (none)
> > > >> OpenGL profile mask: compatibility profile
> > > >>
> > > >> OpenGL ES profile version string: OpenGL ES 3.2 Mesa 19.0.1
> > > >> OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20
> > > >>
> > > >> Any idea/suggestion why is this happening?
> > > >>
> > > >> Thanks!
> > > >> Daniel
> > > > _______________________________________________
> > > > mesa-dev mailing list
> > > > mesa-dev at lists.freedesktop.org
> > > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fmesa-dev&data=02%7C01%7Cthong.thai%40amd.com%7C32c8ff9995e94a08af8408d8387d21bf%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637321457379677028&sdata=GROopf9IjsCCRwP2c6SU2LHQUvSgBQvF8RkqxpSeN68%3D&reserved=0
More information about the mesa-dev
mailing list