Issues with VAAPI and OpenGL
Adrianne Pilleboue
adrianne.pilleboue at wintual.fr
Wed Feb 16 10:23:14 UTC 2022
Thanks a lot for your answer !
Here is the result using GST_DEBUG=*:3,GST_CAPS:5
https://pastebin.com/u7ghfXVN
This part catch my eyes :
0:00:00.171684713 1673 0x7f37000022a0 ERROR vaapivideomemory
gstvaapivideomemory.c:921:allocator_configure_image_info: Cannot create
VA image
0:00:00.171691844 1673 0x7f37000022a0 ERROR vaapipostproc
gstvaapipluginbase.c:608:ensure_srcpad_allocator:<vaapipostproc0> failed
to create src pad's allocator
0:00:00.171695821 1673 0x7f37000022a0 WARN basetransform
gstbasetransform.c:992:gst_base_transform_do_bufferpool:<vaapipostproc0>
Subclass failed to decide allocation
I've also started the pipeline with GST_DEBUG=5 and got a few more line
before this error :
0:00:00.305280570 2401 0x7ff550002120 INFO vaapipostproc
gstvaapipluginbase.c:1196:gst_vaapi_plugin_base_set_gl_context:<vaapipostproc0>
GL context: <glcontextglx0>
0:00:00.305290162 2401 0x7ff550002120 DEBUG video-info
video-info.c:403:gst_video_info_from_caps: parsing caps
video/x-raw(meta:GstVideoGLTextureUploadMeta), format=(string)RGBA,
width=(int)3840, height=(int)2160, framerate=(fraction)24000/1001,
pixel-aspect-ratio=(fraction)1/1, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
interlace-mode=(string)progressive
0:00:00.305294676 2401 0x7ff550002120 DEBUG video-info
video-info.c:525:gst_video_info_from_caps: no colorimetry, using default
0:00:00.305299484 2401 0x7ff550002120 INFO vaapipostproc
gstvaapipluginbase.c:958:gst_vaapi_plugin_base_decide_allocation:<vaapipostproc0>
ignoring non-VAAPI pool: <glbufferpool2>
0:00:00.305307739 2401 0x7ff550002120 DEBUG vaapi
gstvaapisurface.c:423:gst_vaapi_surface_new_full: size 3840x2160, format
RGBA, flags 0x00000000
0:00:00.305322458 2401 0x7ff550002120 DEBUG vaapi
gstvaapisurface.c:212:gst_vaapi_surface_create_full: surface 0x8
0:00:00.305326695 2401 0x7ff550002120 DEBUG vaapi
gstvaapisurface.c:78:gst_vaapi_surface_destroy: surface 0x8
0:00:00.305336482 2401 0x7ff550002120 DEBUG vaapivideomemory
gstvaapivideomemory.c:878:allocator_configure_surface_info:<vaapivideoallocator1>
success to set the surface format RGBA for video format RGBA with native
uploading
0:00:00.305341354 2401 0x7ff550002120 DEBUG vaapi
gstvaapiimage.c:253:gst_vaapi_image_new: format RGBA, size 3840x2160
0:00:00.305345504 2401 0x7ff550002120 DEBUG vaapidisplay
gstvaapidisplay.c:724:ensure_subpicture_formats: 0 subpicture formats
0:00:00.305349621 2401 0x7ff550002120 DEBUG vaapi
gstvaapiimage.c:121:gst_vaapi_image_destroy: image 0xffffffff
I don't understand exactly the difference between
GstVideoGLTextureUploadMeta and DMABuf. Is GstVideoGLTextureUploadMeta
generating a copy ? And why GstVideoGLTextureUploadMeta is not
compatible with NV12 ?
Even if I've no choice but having a copy, I would prefer to have it in
NV12 to reduce the size of the data. NV12 to RGB conversion will then be
done in my shader, and only on the visible part of my video.
On 16.02.2022 08:50, Marianna Smidth Buschle via gstreamer-devel wrote:
>> Hello,
>>
>> In my C++ application, I need to transfer a decoded VASurface in a
>> GLTexture, using glupload and a custom appsink, for 3D rendering
>> purpose.
>> Ideally, I would like a solution with zero-copy texture transfer from
>> VASurface to OpenGL, directly in NV12 format.
>> However, I've difficulties using VAAPI+OpenGL+GStreamer. For example,
>> the following pipeline doesn't work :
>>
>> gst-launch-1.0 urisourcebin uri=file:///home/myusername/myvideo.mp4 !
>> qtdemux ! vaapidecodebin ! glupload ! glcolorconvert ! glimagesink
>>
>> Setting pipeline to PAUSED ...
>> Pipeline is PREROLLING ...
>> Got context from element 'sink': gst.gl.GLDisplay=context,
>> gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0";
>> Got context from element 'vaapipostproc0': gst.vaapi.Display=context,
>> gst.vaapi.Display=(GstVaapiDisplay)"\(GstVaapiDisplayGLX\)\
>> vaapidisplayglx0";
>> Redistribute latency...
>> ERROR: from element
>> /GstPipeline:pipeline0/GstURISourceBin:urisourcebin0/GstTypeFindElement:typefindelement0:
>> Internal data stream error.
>> Additional debug info:
>> gsttypefindelement.c(1228): gst_type_find_element_loop ():
>> /GstPipeline:pipeline0/GstURISourceBin:urisourcebin0/GstTypeFindElement:typefindelement0:
>> streaming stopped, reason not-negotiated (-4)
>> ERROR: pipeline doesn't want to preroll.
>> Setting pipeline to NULL ...
>> Freeing pipeline ...
>>
>> This pipeline is also giving the same result :
>> gst-launch-1.0 urisourcebin uri=file:///home/myusername/myvideo.mp4 !
>> qtdemux ! vaapidecodebin !
>> "video/x-raw(meta:GstVideoGLTextureUploadMeta),format=RGBA" ! glupload
>> !
>> glcolorconvert ! glimagesink
>>
>> Meanwhile, the following pipelines are working, but are usable for my
>> project :
>>
>> gst-launch-1.0 urisourcebin uri=file:///home/myusername/myvideo.mp4 !
>> qtdemux ! vaapidecodebin ! "video/x-raw,format=NV12" ! glupload !
>> glcolorconvert ! glimagesink (non zero-copy)
>> gst-launch-1.0 urisourcebin uri=file:///home/myusername/myvideo.mp4 !
>> qtdemux ! vaapidecodebin ! vaapisink (not using OpenGL)
>>
>> Do you have any idea why the first pipeline is not working ?
>> I'm using Ubuntu 20.04, and the following version of GStreamer :
>> gst-launch-1.0 version 1.16.2
>> GStreamer 1.16.2
>> https://launchpad.net/distros/ubuntu/+source/gstreamer1.0
>
> reason not-negotiated (-4)
>
> So you need to debug the CAPS.
> Try running with GST_DEBUG=*:3,GST_CAPS:5
>
> And see if you can figure out which CAPS are incompatible.
>
> Then you might want to try adding some elements with can convert/parse
> to get the right CAPS, fx 'vaapipostproc' might help.
> But I'm not sure if you can get zero copy.
>
>> Here is the result of gst-inspect on vaapidecodebin, for the src part.
>> I'm wondering why there is not DMABuf capabilities and why
>> GstVideoGLTextureUploadMeta is limited to RGBA format
>
> I can see several issues in Gitlab about DMA, fx:
>
> https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/issues/267
>
> https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/issues/207
>
> https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/issues/105
--
Adrianne PILLEBOUE
Ingénieure informatique chez Wintual
More information about the gstreamer-devel
mailing list