Issues with VAAPI and OpenGL
Victor Jaquez
vjaquez at igalia.com
Wed Feb 16 13:09:56 UTC 2022
On Tue, 15 Feb 2022 at 15:34, Adrianne Pilleboue via gstreamer-devel wrote:
> Hello,
>
> In my C++ application, I need to transfer a decoded VASurface in a
> GLTexture, using glupload and a custom appsink, for 3D rendering purpose.
> Ideally, I would like a solution with zero-copy texture transfer from
> VASurface to OpenGL, directly in NV12 format.
> However, I've difficulties using VAAPI+OpenGL+GStreamer. For example, the
> following pipeline doesn't work :
>
> gst-launch-1.0 urisourcebin uri=file:///home/myusername/myvideo.mp4 !
> qtdemux ! vaapidecodebin ! glupload ! glcolorconvert ! glimagesink
>
> Setting pipeline to PAUSED ...
> Pipeline is PREROLLING ...
> Got context from element 'sink': gst.gl.GLDisplay=context,
> gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0";
> Got context from element 'vaapipostproc0': gst.vaapi.Display=context,
> gst.vaapi.Display=(GstVaapiDisplay)"\(GstVaapiDisplayGLX\)\
> vaapidisplayglx0";
> Redistribute latency...
> ERROR: from element /GstPipeline:pipeline0/GstURISourceBin:urisourcebin0/GstTypeFindElement:typefindelement0:
> Internal data stream error.
> Additional debug info:
> gsttypefindelement.c(1228): gst_type_find_element_loop (): /GstPipeline:pipeline0/GstURISourceBin:urisourcebin0/GstTypeFindElement:typefindelement0:
> streaming stopped, reason not-negotiated (-4)
> ERROR: pipeline doesn't want to preroll.
> Setting pipeline to NULL ...
> Freeing pipeline ...
>
> This pipeline is also giving the same result :
> gst-launch-1.0 urisourcebin uri=file:///home/myusername/myvideo.mp4 !
> qtdemux ! vaapidecodebin !
> "video/x-raw(meta:GstVideoGLTextureUploadMeta),format=RGBA" ! glupload !
> glcolorconvert ! glimagesink
>
> Meanwhile, the following pipelines are working, but are usable for my
> project :
>
> gst-launch-1.0 urisourcebin uri=file:///home/myusername/myvideo.mp4 !
> qtdemux ! vaapidecodebin ! "video/x-raw,format=NV12" ! glupload !
> glcolorconvert ! glimagesink (non zero-copy)
> gst-launch-1.0 urisourcebin uri=file:///home/myusername/myvideo.mp4 !
> qtdemux ! vaapidecodebin ! vaapisink (not using OpenGL)
>
> Do you have any idea why the first pipeline is not working ?
> I'm using Ubuntu 20.04, and the following version of GStreamer :
> gst-launch-1.0 version 1.16.2
> GStreamer 1.16.2
> https://launchpad.net/distros/ubuntu/+source/gstreamer1.0
>
> VAAPI is working fine :
> vainfo
> libva info: VA-API version 1.12.0
> libva info: User environment variable requested driver 'iHD'
> libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
Oh...
You *forgot* to mention that you actually exported
GST_VAAPI_ALL_DRIVERS=1 deactivating the allowed drivers check.
iHD driver *isn't* supported in version 1.16. All the required bits
for supporting it were added in 1.18.
> libva info: Found init function __vaDriverInit_1_12
> libva info: va_openDriver() returns 0
> vainfo: VA-API version: 1.12 (libva 2.6.0)
> vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 21.2.3
> (008fb3b)
> vainfo: Supported profile and entrypoints
> VAProfileNone : VAEntrypointVideoProc
> VAProfileNone : VAEntrypointStats
> VAProfileMPEG2Simple : VAEntrypointVLD
> VAProfileMPEG2Simple : VAEntrypointEncSlice
> VAProfileMPEG2Main : VAEntrypointVLD
> VAProfileMPEG2Main : VAEntrypointEncSlice
> VAProfileH264Main : VAEntrypointVLD
> VAProfileH264Main : VAEntrypointEncSlice
> VAProfileH264Main : VAEntrypointFEI
> VAProfileH264Main : VAEntrypointEncSliceLP
> VAProfileH264High : VAEntrypointVLD
> VAProfileH264High : VAEntrypointEncSlice
> VAProfileH264High : VAEntrypointFEI
> VAProfileH264High : VAEntrypointEncSliceLP
> VAProfileVC1Simple : VAEntrypointVLD
> VAProfileVC1Main : VAEntrypointVLD
> VAProfileVC1Advanced : VAEntrypointVLD
> VAProfileJPEGBaseline : VAEntrypointVLD
> VAProfileJPEGBaseline : VAEntrypointEncPicture
> VAProfileH264ConstrainedBaseline: VAEntrypointVLD
> VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
> VAProfileH264ConstrainedBaseline: VAEntrypointFEI
> VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
> VAProfileVP8Version0_3 : VAEntrypointVLD
> VAProfileVP8Version0_3 : VAEntrypointEncSlice
> VAProfileHEVCMain : VAEntrypointVLD
> VAProfileHEVCMain : VAEntrypointEncSlice
> VAProfileHEVCMain : VAEntrypointFEI
> VAProfileHEVCMain10 : VAEntrypointVLD
> VAProfileHEVCMain10 : VAEntrypointEncSlice
> VAProfileVP9Profile0 : VAEntrypointVLD
> VAProfileVP9Profile2 : VAEntrypointVLD
>
> Here is the result of gst-inspect on vaapidecodebin, for the src part. I'm
> wondering why there is not DMABuf capabilities and why
> GstVideoGLTextureUploadMeta is limited to RGBA format
As I mentioned in the earlier email, DMABuf is only available within EGL.
> Capabilities:
> video/x-raw(memory:VASurface)
> format: { (string)ENCODED, (string)NV12, (string)I420,
> (string)YV12, (string)YUY2, (string)UYVY, (string)Y210, (string)P010_10LE,
> (string)AYUV, (string)Y410 }
> width: [ 1, 2147483647 ]
> height: [ 1, 2147483647 ]
> framerate: [ 0/1, 2147483647/1 ]
> interlace-mode: progressive
> video/x-raw(meta:GstVideoGLTextureUploadMeta)
> format: { (string)RGBA, (string)BGRA }
> width: [ 1, 2147483647 ]
> height: [ 1, 2147483647 ]
> framerate: [ 0/1, 2147483647/1 ]
> interlace-mode: progressive
> video/x-raw
> format: { (string)I420, (string)YV12, (string)YUY2,
> (string)UYVY, (string)AYUV, (string)VUYA, (string)RGBx, (string)BGRx,
> (string)xRGB, (string)xBGR, (string)RGBA, (string)BGRA, (string)ARGB,
> (string)ABGR, (string)RGB, (string)BGR, (string)Y41B, (string)Y42B,
> (string)YVYU, (string)Y444, (string)v210, (string)v216, (string)Y210,
> (string)Y410, (string)NV12, (string)NV21, (string)GRAY8, (string)GRAY16_BE,
> (string)GRAY16_LE, (string)v308, (string)RGB16, (string)BGR16,
> (string)RGB15, (string)BGR15, (string)UYVP, (string)A420, (string)RGB8P,
> (string)YUV9, (string)YVU9, (string)IYU1, (string)ARGB64, (string)AYUV64,
> (string)r210, (string)I420_10BE, (string)I420_10LE, (string)I422_10BE,
> (string)I422_10LE, (string)Y444_10BE, (string)Y444_10LE, (string)GBR,
> (string)GBR_10BE, (string)GBR_10LE, (string)NV16, (string)NV24,
> (string)NV12_64Z32, (string)A420_10BE, (string)A420_10LE, (string)A422_10BE,
> (string)A422_10LE, (string)A444_10BE, (string)A444_10LE, (string)NV61,
> (string)P010_10BE, (string)P010_10LE, (string)IYU2, (string)VYUY,
> (string)GBRA, (string)GBRA_10BE, (string)GBRA_10LE, (string)BGR10A2_LE,
> (string)GBR_12BE, (string)GBR_12LE, (string)GBRA_12BE, (string)GBRA_12LE,
> (string)I420_12BE, (string)I420_12LE, (string)I422_12BE, (string)I422_12LE,
> (string)Y444_12BE, (string)Y444_12LE, (string)GRAY10_LE32,
> (string)NV12_10LE32, (string)NV16_10LE32, (string)NV12_10LE40 }
> width: [ 1, 2147483647 ]
> height: [ 1, 2147483647 ]
> framerate: [ 0/1, 2147483647/1 ]
> interlace-mode: progressive
>
> And the same for the sink of glupload :
> Capabilities:
> video/x-raw(memory:GLMemory, meta:GstVideoOverlayComposition)
> format: { (string)RGBA, (string)BGRA, (string)RGBx,
> (string)BGRx, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR,
> (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)AYUV,
> (string)VUYA, (string)I420, (string)YV12, (string)NV12, (string)NV21,
> (string)YUY2, (string)UYVY, (string)Y41B, (string)Y42B, (string)Y444,
> (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, (string)ARGB64 }
> width: [ 1, 2147483647 ]
> height: [ 1, 2147483647 ]
> framerate: [ 0/1, 2147483647/1 ]
> video/x-raw(memory:DMABuf, meta:GstVideoOverlayComposition)
> format: { (string)RGBA, (string)BGRA, (string)RGBx,
> (string)BGRx, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR,
> (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)AYUV,
> (string)VUYA, (string)I420, (string)YV12, (string)NV12, (string)NV21,
> (string)YUY2, (string)UYVY, (string)Y41B, (string)Y42B, (string)Y444,
> (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, (string)ARGB64 }
> width: [ 1, 2147483647 ]
> height: [ 1, 2147483647 ]
> framerate: [ 0/1, 2147483647/1 ]
> video/x-raw(memory:SystemMemory, meta:GstVideoOverlayComposition)
> format: { (string)RGBA, (string)BGRA, (string)RGBx,
> (string)BGRx, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR,
> (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)AYUV,
> (string)VUYA, (string)I420, (string)YV12, (string)NV12, (string)NV21,
> (string)YUY2, (string)UYVY, (string)Y41B, (string)Y42B, (string)Y444,
> (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, (string)ARGB64 }
> width: [ 1, 2147483647 ]
> height: [ 1, 2147483647 ]
> framerate: [ 0/1, 2147483647/1 ]
> video/x-raw(meta:GstVideoGLTextureUploadMeta,
> meta:GstVideoOverlayComposition)
> format: RGBA
> width: [ 1, 2147483647 ]
> height: [ 1, 2147483647 ]
> framerate: [ 0/1, 2147483647/1 ]
> video/x-raw(memory:GLMemory)
> format: { (string)RGBA, (string)BGRA, (string)RGBx,
> (string)BGRx, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR,
> (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)AYUV,
> (string)VUYA, (string)I420, (string)YV12, (string)NV12, (string)NV21,
> (string)YUY2, (string)UYVY, (string)Y41B, (string)Y42B, (string)Y444,
> (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, (string)ARGB64 }
> width: [ 1, 2147483647 ]
> height: [ 1, 2147483647 ]
> framerate: [ 0/1, 2147483647/1 ]
> video/x-raw(memory:DMABuf)
> format: { (string)RGBA, (string)BGRA, (string)RGBx,
> (string)BGRx, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR,
> (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)AYUV,
> (string)VUYA, (string)I420, (string)YV12, (string)NV12, (string)NV21,
> (string)YUY2, (string)UYVY, (string)Y41B, (string)Y42B, (string)Y444,
> (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, (string)ARGB64 }
> width: [ 1, 2147483647 ]
> height: [ 1, 2147483647 ]
> framerate: [ 0/1, 2147483647/1 ]
> video/x-raw
> format: { (string)RGBA, (string)BGRA, (string)RGBx,
> (string)BGRx, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR,
> (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)AYUV,
> (string)VUYA, (string)I420, (string)YV12, (string)NV12, (string)NV21,
> (string)YUY2, (string)UYVY, (string)Y41B, (string)Y42B, (string)Y444,
> (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, (string)ARGB64 }
> width: [ 1, 2147483647 ]
> height: [ 1, 2147483647 ]
> framerate: [ 0/1, 2147483647/1 ]
> video/x-raw(meta:GstVideoGLTextureUploadMeta)
> format: RGBA
> width: [ 1, 2147483647 ]
> height: [ 1, 2147483647 ]
> framerate: [ 0/1, 2147483647/1 ]
>
> Here is also some debug stuff with GST_DEBUG=4 :
> https://pastebin.com/ajza3Si5
>
> Cheers
>
More information about the gstreamer-devel
mailing list