Issues with VAAPI and OpenGL
Adrianne Pilleboue
adrianne.pilleboue at wintual.fr
Tue Feb 22 14:39:16 UTC 2022
Thanks a lot, the pipeline is now working fine with GStreamer 1.20
On 16.02.2022 14:09, Victor Jaquez via gstreamer-devel wrote:
> On Tue, 15 Feb 2022 at 15:34, Adrianne Pilleboue via gstreamer-devel
> wrote:
>> Hello,
>>
>> In my C++ application, I need to transfer a decoded VASurface in a
>> GLTexture, using glupload and a custom appsink, for 3D rendering
>> purpose.
>> Ideally, I would like a solution with zero-copy texture transfer from
>> VASurface to OpenGL, directly in NV12 format.
>> However, I've difficulties using VAAPI+OpenGL+GStreamer. For example,
>> the
>> following pipeline doesn't work :
>>
>> gst-launch-1.0 urisourcebin uri=file:///home/myusername/myvideo.mp4 !
>> qtdemux ! vaapidecodebin ! glupload ! glcolorconvert ! glimagesink
>>
>> Setting pipeline to PAUSED ...
>> Pipeline is PREROLLING ...
>> Got context from element 'sink': gst.gl.GLDisplay=context,
>> gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0";
>> Got context from element 'vaapipostproc0': gst.vaapi.Display=context,
>> gst.vaapi.Display=(GstVaapiDisplay)"\(GstVaapiDisplayGLX\)\
>> vaapidisplayglx0";
>> Redistribute latency...
>> ERROR: from element
>> /GstPipeline:pipeline0/GstURISourceBin:urisourcebin0/GstTypeFindElement:typefindelement0:
>> Internal data stream error.
>> Additional debug info:
>> gsttypefindelement.c(1228): gst_type_find_element_loop ():
>> /GstPipeline:pipeline0/GstURISourceBin:urisourcebin0/GstTypeFindElement:typefindelement0:
>> streaming stopped, reason not-negotiated (-4)
>> ERROR: pipeline doesn't want to preroll.
>> Setting pipeline to NULL ...
>> Freeing pipeline ...
>>
>> This pipeline is also giving the same result :
>> gst-launch-1.0 urisourcebin uri=file:///home/myusername/myvideo.mp4 !
>> qtdemux ! vaapidecodebin !
>> "video/x-raw(meta:GstVideoGLTextureUploadMeta),format=RGBA" ! glupload
>> !
>> glcolorconvert ! glimagesink
>>
>> Meanwhile, the following pipelines are working, but are usable for my
>> project :
>>
>> gst-launch-1.0 urisourcebin uri=file:///home/myusername/myvideo.mp4 !
>> qtdemux ! vaapidecodebin ! "video/x-raw,format=NV12" ! glupload !
>> glcolorconvert ! glimagesink (non zero-copy)
>> gst-launch-1.0 urisourcebin uri=file:///home/myusername/myvideo.mp4 !
>> qtdemux ! vaapidecodebin ! vaapisink (not using OpenGL)
>>
>> Do you have any idea why the first pipeline is not working ?
>> I'm using Ubuntu 20.04, and the following version of GStreamer :
>> gst-launch-1.0 version 1.16.2
>> GStreamer 1.16.2
>> https://launchpad.net/distros/ubuntu/+source/gstreamer1.0
>>
>> VAAPI is working fine :
>> vainfo
>> libva info: VA-API version 1.12.0
>> libva info: User environment variable requested driver 'iHD'
>> libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
>
> Oh...
>
> You *forgot* to mention that you actually exported
> GST_VAAPI_ALL_DRIVERS=1 deactivating the allowed drivers check.
>
> iHD driver *isn't* supported in version 1.16. All the required bits
> for supporting it were added in 1.18.
>
>> libva info: Found init function __vaDriverInit_1_12
>> libva info: va_openDriver() returns 0
>> vainfo: VA-API version: 1.12 (libva 2.6.0)
>> vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics -
>> 21.2.3
>> (008fb3b)
>> vainfo: Supported profile and entrypoints
>> VAProfileNone : VAEntrypointVideoProc
>> VAProfileNone : VAEntrypointStats
>> VAProfileMPEG2Simple : VAEntrypointVLD
>> VAProfileMPEG2Simple : VAEntrypointEncSlice
>> VAProfileMPEG2Main : VAEntrypointVLD
>> VAProfileMPEG2Main : VAEntrypointEncSlice
>> VAProfileH264Main : VAEntrypointVLD
>> VAProfileH264Main : VAEntrypointEncSlice
>> VAProfileH264Main : VAEntrypointFEI
>> VAProfileH264Main : VAEntrypointEncSliceLP
>> VAProfileH264High : VAEntrypointVLD
>> VAProfileH264High : VAEntrypointEncSlice
>> VAProfileH264High : VAEntrypointFEI
>> VAProfileH264High : VAEntrypointEncSliceLP
>> VAProfileVC1Simple : VAEntrypointVLD
>> VAProfileVC1Main : VAEntrypointVLD
>> VAProfileVC1Advanced : VAEntrypointVLD
>> VAProfileJPEGBaseline : VAEntrypointVLD
>> VAProfileJPEGBaseline : VAEntrypointEncPicture
>> VAProfileH264ConstrainedBaseline: VAEntrypointVLD
>> VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
>> VAProfileH264ConstrainedBaseline: VAEntrypointFEI
>> VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
>> VAProfileVP8Version0_3 : VAEntrypointVLD
>> VAProfileVP8Version0_3 : VAEntrypointEncSlice
>> VAProfileHEVCMain : VAEntrypointVLD
>> VAProfileHEVCMain : VAEntrypointEncSlice
>> VAProfileHEVCMain : VAEntrypointFEI
>> VAProfileHEVCMain10 : VAEntrypointVLD
>> VAProfileHEVCMain10 : VAEntrypointEncSlice
>> VAProfileVP9Profile0 : VAEntrypointVLD
>> VAProfileVP9Profile2 : VAEntrypointVLD
>>
>> Here is the result of gst-inspect on vaapidecodebin, for the src part.
>> I'm
>> wondering why there is not DMABuf capabilities and why
>> GstVideoGLTextureUploadMeta is limited to RGBA format
>
> As I mentioned in the earlier email, DMABuf is only available within
> EGL.
>
>> Capabilities:
>> video/x-raw(memory:VASurface)
>> format: { (string)ENCODED, (string)NV12,
>> (string)I420,
>> (string)YV12, (string)YUY2, (string)UYVY, (string)Y210,
>> (string)P010_10LE,
>> (string)AYUV, (string)Y410 }
>> width: [ 1, 2147483647 ]
>> height: [ 1, 2147483647 ]
>> framerate: [ 0/1, 2147483647/1 ]
>> interlace-mode: progressive
>> video/x-raw(meta:GstVideoGLTextureUploadMeta)
>> format: { (string)RGBA, (string)BGRA }
>> width: [ 1, 2147483647 ]
>> height: [ 1, 2147483647 ]
>> framerate: [ 0/1, 2147483647/1 ]
>> interlace-mode: progressive
>> video/x-raw
>> format: { (string)I420, (string)YV12, (string)YUY2,
>> (string)UYVY, (string)AYUV, (string)VUYA, (string)RGBx, (string)BGRx,
>> (string)xRGB, (string)xBGR, (string)RGBA, (string)BGRA, (string)ARGB,
>> (string)ABGR, (string)RGB, (string)BGR, (string)Y41B, (string)Y42B,
>> (string)YVYU, (string)Y444, (string)v210, (string)v216, (string)Y210,
>> (string)Y410, (string)NV12, (string)NV21, (string)GRAY8,
>> (string)GRAY16_BE,
>> (string)GRAY16_LE, (string)v308, (string)RGB16, (string)BGR16,
>> (string)RGB15, (string)BGR15, (string)UYVP, (string)A420,
>> (string)RGB8P,
>> (string)YUV9, (string)YVU9, (string)IYU1, (string)ARGB64,
>> (string)AYUV64,
>> (string)r210, (string)I420_10BE, (string)I420_10LE, (string)I422_10BE,
>> (string)I422_10LE, (string)Y444_10BE, (string)Y444_10LE, (string)GBR,
>> (string)GBR_10BE, (string)GBR_10LE, (string)NV16, (string)NV24,
>> (string)NV12_64Z32, (string)A420_10BE, (string)A420_10LE,
>> (string)A422_10BE,
>> (string)A422_10LE, (string)A444_10BE, (string)A444_10LE, (string)NV61,
>> (string)P010_10BE, (string)P010_10LE, (string)IYU2, (string)VYUY,
>> (string)GBRA, (string)GBRA_10BE, (string)GBRA_10LE,
>> (string)BGR10A2_LE,
>> (string)GBR_12BE, (string)GBR_12LE, (string)GBRA_12BE,
>> (string)GBRA_12LE,
>> (string)I420_12BE, (string)I420_12LE, (string)I422_12BE,
>> (string)I422_12LE,
>> (string)Y444_12BE, (string)Y444_12LE, (string)GRAY10_LE32,
>> (string)NV12_10LE32, (string)NV16_10LE32, (string)NV12_10LE40 }
>> width: [ 1, 2147483647 ]
>> height: [ 1, 2147483647 ]
>> framerate: [ 0/1, 2147483647/1 ]
>> interlace-mode: progressive
>>
>> And the same for the sink of glupload :
>> Capabilities:
>> video/x-raw(memory:GLMemory, meta:GstVideoOverlayComposition)
>> format: { (string)RGBA, (string)BGRA, (string)RGBx,
>> (string)BGRx, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR,
>> (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)AYUV,
>> (string)VUYA, (string)I420, (string)YV12, (string)NV12, (string)NV21,
>> (string)YUY2, (string)UYVY, (string)Y41B, (string)Y42B, (string)Y444,
>> (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, (string)ARGB64 }
>> width: [ 1, 2147483647 ]
>> height: [ 1, 2147483647 ]
>> framerate: [ 0/1, 2147483647/1 ]
>> video/x-raw(memory:DMABuf, meta:GstVideoOverlayComposition)
>> format: { (string)RGBA, (string)BGRA, (string)RGBx,
>> (string)BGRx, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR,
>> (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)AYUV,
>> (string)VUYA, (string)I420, (string)YV12, (string)NV12, (string)NV21,
>> (string)YUY2, (string)UYVY, (string)Y41B, (string)Y42B, (string)Y444,
>> (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, (string)ARGB64 }
>> width: [ 1, 2147483647 ]
>> height: [ 1, 2147483647 ]
>> framerate: [ 0/1, 2147483647/1 ]
>> video/x-raw(memory:SystemMemory,
>> meta:GstVideoOverlayComposition)
>> format: { (string)RGBA, (string)BGRA, (string)RGBx,
>> (string)BGRx, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR,
>> (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)AYUV,
>> (string)VUYA, (string)I420, (string)YV12, (string)NV12, (string)NV21,
>> (string)YUY2, (string)UYVY, (string)Y41B, (string)Y42B, (string)Y444,
>> (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, (string)ARGB64 }
>> width: [ 1, 2147483647 ]
>> height: [ 1, 2147483647 ]
>> framerate: [ 0/1, 2147483647/1 ]
>> video/x-raw(meta:GstVideoGLTextureUploadMeta,
>> meta:GstVideoOverlayComposition)
>> format: RGBA
>> width: [ 1, 2147483647 ]
>> height: [ 1, 2147483647 ]
>> framerate: [ 0/1, 2147483647/1 ]
>> video/x-raw(memory:GLMemory)
>> format: { (string)RGBA, (string)BGRA, (string)RGBx,
>> (string)BGRx, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR,
>> (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)AYUV,
>> (string)VUYA, (string)I420, (string)YV12, (string)NV12, (string)NV21,
>> (string)YUY2, (string)UYVY, (string)Y41B, (string)Y42B, (string)Y444,
>> (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, (string)ARGB64 }
>> width: [ 1, 2147483647 ]
>> height: [ 1, 2147483647 ]
>> framerate: [ 0/1, 2147483647/1 ]
>> video/x-raw(memory:DMABuf)
>> format: { (string)RGBA, (string)BGRA, (string)RGBx,
>> (string)BGRx, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR,
>> (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)AYUV,
>> (string)VUYA, (string)I420, (string)YV12, (string)NV12, (string)NV21,
>> (string)YUY2, (string)UYVY, (string)Y41B, (string)Y42B, (string)Y444,
>> (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, (string)ARGB64 }
>> width: [ 1, 2147483647 ]
>> height: [ 1, 2147483647 ]
>> framerate: [ 0/1, 2147483647/1 ]
>> video/x-raw
>> format: { (string)RGBA, (string)BGRA, (string)RGBx,
>> (string)BGRx, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR,
>> (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)AYUV,
>> (string)VUYA, (string)I420, (string)YV12, (string)NV12, (string)NV21,
>> (string)YUY2, (string)UYVY, (string)Y41B, (string)Y42B, (string)Y444,
>> (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, (string)ARGB64 }
>> width: [ 1, 2147483647 ]
>> height: [ 1, 2147483647 ]
>> framerate: [ 0/1, 2147483647/1 ]
>> video/x-raw(meta:GstVideoGLTextureUploadMeta)
>> format: RGBA
>> width: [ 1, 2147483647 ]
>> height: [ 1, 2147483647 ]
>> framerate: [ 0/1, 2147483647/1 ]
>>
>> Here is also some debug stuff with GST_DEBUG=4 :
>> https://pastebin.com/ajza3Si5
>>
>> Cheers
>>
--
Adrianne PILLEBOUE
Ingénieure informatique chez Wintual
More information about the gstreamer-devel
mailing list