<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> </head> <body> > I don't understand exactly the difference between GstVideoGLTextureUploadMeta and DMABuf. Is GstVideoGLTextureUploadMeta generating a copy ? And why GstVideoGLTextureUploadMeta is not compatible with NV12 ? I cant help you with that unfortunately, I know pretty much nothing about GL and DMA and have only used VAAPI sparsely. > Here is the result using GST_DEBUG=*:3,GST_CAPS:5 What I generally look for with that command is more something like this: gst-launch-1.0 v4l2src ! pnmenc ! fakesink --gst-debug=*:3,GST_CAPS:5 0:00:00.129351972 97934 0x564cb7466e30 DEBUG GST_CAPS gstutils.c:3109:gst_pad_query_caps:<pnmenc0:sink> query returned video/x-raw, format=(string){ RGB, GRAY8, GRAY16_BE, GRAY16_LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ] 0:00:00.129480601 97934 0x564cb7466e30 DEBUG GST_CAPS gstpad.c:2284:gst_pad_link_check_compatible_unlocked:<pnmenc0:sink> sink caps video/x-raw, format=(string){ RGB, GRAY8, GRAY16_BE, GRAY16_LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ] 0:00:00.131267810 97934 0x564cb7452f60 DEBUG GST_CAPS gstutils.c:3109:gst_pad_query_caps:<v4l2src0:src> query returned video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)10/1; video/x-raw, format=(string)YUY2, width=(int)960, height=(int)540, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)15/1; video/x-raw, format=(string)YUY2, width=(int)848, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1; video/x-raw, format=(string)YUY2, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1; video/x-raw, format=(string)YUY2, width=(int)640, height=(int)360, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1; video/x-raw, format=(string)YUY2, width=(int)424, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1; video/x-raw, format=(string)YUY2, width=(int)352, height=(int)288, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1; video/x-raw, format=(string)YUY2, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1; video/x-raw, format=(string)YUY2, width=(int)320, height=(int)180, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1; image/jpeg, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1; image/jpeg, width=(int)960, height=(int)540, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1; image/jpeg, width=(int)848, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1; image/jpeg, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1; image/jpeg, width=(int)640, height=(int)360, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1; image/jpeg, width=(int)424, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1; image/jpeg, width=(int)352, height=(int)288, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1; image/jpeg, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1; image/jpeg, width=(int)320, height=(int)180, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1 0:00:00.131425569 97934 0x564cb7452f60 DEBUG GST_CAPS gstutils.c:3155:gst_pad_peer_query_caps:<v4l2src0:src> peer query returned video/x-raw, format=(string){ RGB, GRAY8, GRAY16_BE, GRAY16_LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ] 0:00:00.131476694 97934 0x564cb7452f60 WARN basesrc gstbasesrc.c:3127:gst_base_src_loop:<v4l2src0> error: Internal data stream error. 0:00:00.131495621 97934 0x564cb7452f60 WARN basesrc gstbasesrc.c:3127:gst_base_src_loop:<v4l2src0> error: streaming stopped, reason not-negotiated (-4) So you can see that 'pnmenc' wants one of these: RGB, GRAY8, GRAY16_BE, GRAY16_LE But my v4l2src only provides YUY2, so it can't work without a videoconvert in between. Alternatively you can try simplifying you pipeline and seeing where it fails: gst-launch-1.0 urisourcebin uri=<a class="moz-txt-link-freetext" href="file:///home/myusername/myvideo.mp4">file:///home/myusername/myvideo.mp4</a> ! qtdemux ! vaapidecodebin ! glupload ! glcolorconvert ! glimagesink gst-launch-1.0 urisourcebin uri=<a class="moz-txt-link-freetext" href="file:///home/myusername/myvideo.mp4">file:///home/myusername/myvideo.mp4</a> ! qtdemux ! vaapidecodebin ! glupload ! glcolorconvert ! fakesink gst-launch-1.0 urisourcebin uri=<a class="moz-txt-link-freetext" href="file:///home/myusername/myvideo.mp4">file:///home/myusername/myvideo.mp4</a> ! qtdemux ! vaapidecodebin ! glupload ! fakesink gst-launch-1.0 urisourcebin uri=<a class="moz-txt-link-freetext" href="file:///home/myusername/myvideo.mp4">file:///home/myusername/myvideo.mp4</a> ! qtdemux ! vaapidecodebin ! fakesink And swapping the vaapidecodebin to the individual elements to see if that is the problem gst-launch-1.0 urisourcebin uri=<a class="moz-txt-link-freetext" href="file:///home/myusername/myvideo.mp4">file:///home/myusername/myvideo.mp4</a> ! qtdemux ! <span style="color: rgb(0, 0, 0); font-family: cantarell, sans-serif; font-size: medium; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">vaapidecode ! vaapipostproc ! <span style="color: rgb(0, 0, 0); font-family: cantarell, sans-serif; font-size: medium; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">fakesink <span style="color: rgb(0, 0, 0); font-family: cantarell, sans-serif; font-size: medium; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">gst-launch-1.0 urisourcebin uri=<a class="moz-txt-link-freetext" href="file:///home/myusername/myvideo.mp4">file:///home/myusername/myvideo.mp4</a> ! qtdemux ! <span style="color: rgb(0, 0, 0); font-family: cantarell, sans-serif; font-size: medium; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">vaapidecode ! <span style="color: rgb(0, 0, 0); font-family: cantarell, sans-serif; font-size: medium; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">fakesink gst-launch-1.0 urisourcebin uri=<a class="moz-txt-link-freetext" href="file:///home/myusername/myvideo.mp4">file:///home/myusername/myvideo.mp4</a> ! qtdemux ! <span style="color: rgb(0, 0, 0); font-family: cantarell, sans-serif; font-size: medium; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">vaapidecode ! vaapipostproc ! <span style="color: rgb(0, 0, 0); font-family: cantarell, sans-serif; font-size: medium; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">glupload ! ... <span style="color: rgb(0, 0, 0); font-family: cantarell, sans-serif; font-size: medium; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;"> <a class="moz-txt-link-freetext" href="https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-vaapi-plugins/html/gstreamer-vaapi-plugins-vaapidecodebin.html">https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-vaapi-plugins/html/gstreamer-vaapi-plugins-vaapidecodebin.html</a> <p style="max-width: 60em; color: rgb(0, 0, 0); font-family: cantarell, sans-serif; font-size: medium; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">vaapidecodebin is similar <a class="link" href="https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-vaapi-plugins/html/gstreamer-vaapi-plugins-vaapidecode.html#GstVaapiDecode" style="text-decoration: none; color: rgb(52, 101, 164);">GstVaapiDecode</a>, but it is composed by the vaapidecode, a <a href="https://gstreamer.freedesktop.org/usr/share/gtk-doc/html/gstreamer-plugins-1.0gstreamer-plugins-queue.html#GstQueue-struct" style="text-decoration: none; color: rgb(52, 101, 164);">GstQueue</a>, and the <a class="link" href="https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-vaapi-plugins/html/gstreamer-vaapi-plugins-vaapipostproc.html#GstVaapiPostproc" style="text-decoration: none; color: rgb(52, 101, 164);">GstVaapiPostproc</a>, if it is available and functional in the setup. <p style="max-width: 60em; color: rgb(0, 0, 0); font-family: cantarell, sans-serif; font-size: medium; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">It offers the functionality of <a class="link" href="https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-vaapi-plugins/html/gstreamer-vaapi-plugins-vaapidecode.html#GstVaapiDecode" style="text-decoration: none; color: rgb(52, 101, 164);">GstVaapiDecode</a> and the many options of <a class="link" href="https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-vaapi-plugins/html/gstreamer-vaapi-plugins-vaapipostproc.html#GstVaapiPostproc" style="text-decoration: none; color: rgb(52, 101, 164);">GstVaapiPostproc</a>. <div class="moz-cite-prefix">On 16.02.2022 11.23, Adrianne Pilleboue wrote: </div> <blockquote type="cite" cite="mid:16617806a4787109e6df71efc1bdb0fd@wintual.fr">Thanks a lot for your answer ! Here is the result using GST_DEBUG=*:3,GST_CAPS:5 <a class="moz-txt-link-freetext" href="https://pastebin.com/u7ghfXVN">https://pastebin.com/u7ghfXVN</a> This part catch my eyes : 0:00:00.171684713 1673 0x7f37000022a0 ERROR vaapivideomemory gstvaapivideomemory.c:921:allocator_configure_image_info: Cannot create VA image 0:00:00.171691844 1673 0x7f37000022a0 ERROR vaapipostproc gstvaapipluginbase.c:608:ensure_srcpad_allocator:<vaapipostproc0> failed to create src pad's allocator 0:00:00.171695821 1673 0x7f37000022a0 WARN basetransform gstbasetransform.c:992:gst_base_transform_do_bufferpool:<vaapipostproc0> Subclass failed to decide allocation I've also started the pipeline with GST_DEBUG=5 and got a few more line before this error : 0:00:00.305280570 2401 0x7ff550002120 INFO vaapipostproc gstvaapipluginbase.c:1196:gst_vaapi_plugin_base_set_gl_context:<vaapipostproc0> GL context: <glcontextglx0> 0:00:00.305290162 2401 0x7ff550002120 DEBUG video-info video-info.c:403:gst_video_info_from_caps: parsing caps video/x-raw(meta:GstVideoGLTextureUploadMeta), format=(string)RGBA, width=(int)3840, height=(int)2160, framerate=(fraction)24000/1001, pixel-aspect-ratio=(fraction)1/1, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, interlace-mode=(string)progressive 0:00:00.305294676 2401 0x7ff550002120 DEBUG video-info video-info.c:525:gst_video_info_from_caps: no colorimetry, using default 0:00:00.305299484 2401 0x7ff550002120 INFO vaapipostproc gstvaapipluginbase.c:958:gst_vaapi_plugin_base_decide_allocation:<vaapipostproc0> ignoring non-VAAPI pool: <glbufferpool2> 0:00:00.305307739 2401 0x7ff550002120 DEBUG vaapi gstvaapisurface.c:423:gst_vaapi_surface_new_full: size 3840x2160, format RGBA, flags 0x00000000 0:00:00.305322458 2401 0x7ff550002120 DEBUG vaapi gstvaapisurface.c:212:gst_vaapi_surface_create_full: surface 0x8 0:00:00.305326695 2401 0x7ff550002120 DEBUG vaapi gstvaapisurface.c:78:gst_vaapi_surface_destroy: surface 0x8 0:00:00.305336482 2401 0x7ff550002120 DEBUG vaapivideomemory gstvaapivideomemory.c:878:allocator_configure_surface_info:<vaapivideoallocator1> success to set the surface format RGBA for video format RGBA with native uploading 0:00:00.305341354 2401 0x7ff550002120 DEBUG vaapi gstvaapiimage.c:253:gst_vaapi_image_new: format RGBA, size 3840x2160 0:00:00.305345504 2401 0x7ff550002120 DEBUG vaapidisplay gstvaapidisplay.c:724:ensure_subpicture_formats: 0 subpicture formats 0:00:00.305349621 2401 0x7ff550002120 DEBUG vaapi gstvaapiimage.c:121:gst_vaapi_image_destroy: image 0xffffffff I don't understand exactly the difference between GstVideoGLTextureUploadMeta and DMABuf. Is GstVideoGLTextureUploadMeta generating a copy ? And why GstVideoGLTextureUploadMeta is not compatible with NV12 ? Even if I've no choice but having a copy, I would prefer to have it in NV12 to reduce the size of the data. NV12 to RGB conversion will then be done in my shader, and only on the visible part of my video. On 16.02.2022 08:50, Marianna Smidth Buschle via gstreamer-devel wrote: <blockquote type="cite"> <blockquote type="cite">Hello, In my C++ application, I need to transfer a decoded VASurface in a GLTexture, using glupload and a custom appsink, for 3D rendering purpose. Ideally, I would like a solution with zero-copy texture transfer from VASurface to OpenGL, directly in NV12 format. However, I've difficulties using VAAPI+OpenGL+GStreamer. For example, the following pipeline doesn't work : gst-launch-1.0 urisourcebin uri=<a class="moz-txt-link-freetext" href="file:///home/myusername/myvideo.mp4">file:///home/myusername/myvideo.mp4</a> ! qtdemux ! vaapidecodebin ! glupload ! glcolorconvert ! glimagesink Setting pipeline to PAUSED ... Pipeline is PREROLLING ... Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0"; Got context from element 'vaapipostproc0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"\(GstVaapiDisplayGLX\)\ vaapidisplayglx0"; Redistribute latency... ERROR: from element /GstPipeline:pipeline0/GstURISourceBin:urisourcebin0/GstTypeFindElement:typefindelement0: Internal data stream error. Additional debug info: gsttypefindelement.c(1228): gst_type_find_element_loop (): /GstPipeline:pipeline0/GstURISourceBin:urisourcebin0/GstTypeFindElement:typefindelement0: streaming stopped, reason not-negotiated (-4) ERROR: pipeline doesn't want to preroll. Setting pipeline to NULL ... Freeing pipeline ... This pipeline is also giving the same result : gst-launch-1.0 urisourcebin uri=<a class="moz-txt-link-freetext" href="file:///home/myusername/myvideo.mp4">file:///home/myusername/myvideo.mp4</a> ! qtdemux ! vaapidecodebin ! "video/x-raw(meta:GstVideoGLTextureUploadMeta),format=RGBA" ! glupload ! glcolorconvert ! glimagesink Meanwhile, the following pipelines are working, but are usable for my project : gst-launch-1.0 urisourcebin uri=<a class="moz-txt-link-freetext" href="file:///home/myusername/myvideo.mp4">file:///home/myusername/myvideo.mp4</a> ! qtdemux ! vaapidecodebin ! "video/x-raw,format=NV12" ! glupload ! glcolorconvert ! glimagesink (non zero-copy) gst-launch-1.0 urisourcebin uri=<a class="moz-txt-link-freetext" href="file:///home/myusername/myvideo.mp4">file:///home/myusername/myvideo.mp4</a> ! qtdemux ! vaapidecodebin ! vaapisink (not using OpenGL) Do you have any idea why the first pipeline is not working ? I'm using Ubuntu 20.04, and the following version of GStreamer : gst-launch-1.0 version 1.16.2 GStreamer 1.16.2 <a class="moz-txt-link-freetext" href="https://launchpad.net/distros/ubuntu/+source/gstreamer1.0">https://launchpad.net/distros/ubuntu/+source/gstreamer1.0</a> </blockquote> reason not-negotiated (-4) So you need to debug the CAPS. Try running with GST_DEBUG=*:3,GST_CAPS:5 And see if you can figure out which CAPS are incompatible. Then you might want to try adding some elements with can convert/parse to get the right CAPS, fx 'vaapipostproc' might help. But I'm not sure if you can get zero copy. <blockquote type="cite">Here is the result of gst-inspect on vaapidecodebin, for the src part. I'm wondering why there is not DMABuf capabilities and why GstVideoGLTextureUploadMeta is limited to RGBA format </blockquote> I can see several issues in Gitlab about DMA, fx: <a class="moz-txt-link-freetext" href="https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/issues/267">https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/issues/267</a> <a class="moz-txt-link-freetext" href="https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/issues/207">https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/issues/207</a> <a class="moz-txt-link-freetext" href="https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/issues/105">https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/issues/105</a> </blockquote> </blockquote> <pre class="moz-signature" cols="72">-- Best regards / Med venlig hilsen “Marianna Smidth Buschle”</pre> </body> </html>