[External] AW: gstreamer videoconvert support gpu memory buffer

Wed Aug 18 08:17:48 UTC 2021

Hi,

As I can see glcolorconvert expect sink and src as video/x-raw(memory:GLMemory), which is not same in my case as input for me is x-raw and o/p is x-raw(memory:GBM)

In my case I am trying to access usb camera using /dev/video3 so input to videoconvert/or any other plugin would be

video/x-raw, width=(int)1280, height=(int)720, framerate=(fraction)15/2, format=(string)YUY2

and the aim is to convert using customized videoconvert/other element.

video/x-raw(memory:GBM), width=(int)1280, height=(int)720, framerate=(fraction)15/2, format=(string)NV12

the videoconvert customized is aimed to perform

  1.  format conversion from YUY2 to NV12
  2.  frame buffers from video/x-raw(system memory) to video/x-raw(memory:GBM)(gpu backed memory)

Here GBM buffers are gpu backed memory which I am setting up in transform_frame of videoconvert.

Above necessity is as qtimletflite requires sink input as NV12 format and video/x-raw(memory:GBM)

Playback Pipeline is as follows. Below videoconvert is customized version

gst-launch-1.0 -v v4l2src device=/dev/video3 num-buffers=1 ! video/x-raw,width=1280,height=720,framerate=15/2,format=YUY2 ! videoconvert ! qtimletflite config=/data/misc/camera/mle_tflite.config model=/data/misc/camera/detect.tflite labels=/data/misc/camera/labelmap.txt postprocessing=detection ! waylandsink

Regards
Gaurav
From: gstreamer-devel <gstreamer-devel-bounces at lists.freedesktop.org> On Behalf Of Thornton, Keith via gstreamer-devel
Sent: Wednesday, August 18, 2021 1:14 PM
To: Discussion of the development of and with GStreamer <gstreamer-devel at lists.freedesktop.org>
Cc: Thornton, Keith <keith.thornton at zeiss.com>
Subject: [External] AW: gstreamer videoconvert support gpu memory buffer

CAUTION: This email originated from outside of the organization. This message might not be safe, use caution in opening it. If in doubt, do not open the attachment nor links in the message.

Hi,
does glcolorconvert not do what you need?
Gruesse

Von: gstreamer-devel <gstreamer-devel-bounces at lists.freedesktop.org<mailto:gstreamer-devel-bounces at lists.freedesktop.org>> Im Auftrag von Gaurav Vohra via gstreamer-devel
Gesendet: Mittwoch, 18. August 2021 08:18
An: gstreamer-devel at lists.freedesktop.org<mailto:gstreamer-devel at lists.freedesktop.org>
Cc: Gaurav Vohra <Gaurav.Vohra at einfochips.com<mailto:Gaurav.Vohra at einfochips.com>>
Betreff: gstreamer videoconvert support gpu memory buffer

Hello,

I am working to customize gstreamer plugin to support

working to modify gstreamer plugin - videoconvert get obtain output data allocated as gpu memory allocated buffers.
As part of first step have modified the plugin to update the src template to include the gpu memory caps.
Allocation for gpu memory is done is transform_frame, but before that being called stream error and negotition is failing
Commented part
// #define CSP_VIDEO_CAPS GST_VIDEO_CAPS_MAKE (GST_VIDEO_FORMATS_ALL) ";" \
GST_VIDEO_CAPS_MAKE_WITH_FEATURES ("ANY", GST_VIDEO_FORMATS_ALL)
Modified part
#define CSP_VIDEO_CAPS_SRC GST_VIDEO_CAPS_MAKE (GST_VIDEO_FORMATS_ALL) ";" \
GST_VIDEO_CAPS_MAKE_WITH_FEATURES ("memory:GBM", GST_VIDEO_FORMATS_ALL)

//#define CSP_VIDEO_CAPS_SRC GST_VIDEO_CAPS_MAKE_WITH_FEATURES ("memory:GBM", GST_VIDEO_FORMATS_ALL)
static GstStaticPadTemplate gst_video_convert_src_template =
GST_STATIC_PAD_TEMPLATE ("src",
    GST_PAD_SRC,
    GST_PAD_ALWAYS,
    GST_STATIC_CAPS (CSP_VIDEO_CAPS_SRC)
    );
While trying to run the playback pipeline getting errors related to negotitation and stream error like below
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../../../../gstreamer-1.14.4/libs/gst/base/gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)

Further debugging from logs suggest intersect of capsfilter input and the caps filter is coming as EMPTY

0:00:06.974521920  4320  0x10cde60 DEBUG             capsfilter gstcapsfilter.c:296:gst_capsfilter_transform_caps:<capsfilter0> input:     video/x-raw(memory:GBM), format=(string)NV12, width=(int)[ 1, 32767 ], height=(int)[ 1, 32767 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw(memory:GBM), width=(int)[ 1, 32767 ], height=(int)[ 1, 32767 ], framerate=(fraction)[ 0/1, 2147483647/1 ], format=(string){ I420, YV12, YUY2, UYVY, AYUV, RGBx, BGRx, xRGB, xBGR, RGBA, BGRA, ARGB, ABGR, RGB, BGR, Y41B, Y42B, YVYU, Y444, v210, v216, NV12, NV21, GRAY8, GRAY16_BE, GRAY16_LE, v308, RGB16, BGR16, RGB15, BGR15, UYVP, A420, RGB8P, YUV9, YVU9, IYU1, ARGB64, AYUV64, r210, I420_10BE, I420_10LE, I422_10BE, I422_10LE, Y444_10BE, Y444_10LE, GBR, GBR_10BE, GBR_10LE, NV16, NV24, NV12_64Z32, A420_10BE, A420_10LE, A422_10BE, A422_10LE, A444_10BE, A444_10LE, NV61, P010_10BE, P010_10LE, IYU2, VYUY, GBRA, GBRA_10BE, GBRA_10LE, GBR_12BE, GBR_12LE, GBRA_12BE, GBRA_12LE, I420_12BE, I420_12LE, I422_12BE, I422_12LE, Y444_12BE, Y444_12LE, GRAY10_LE32, NV12_10LE32, NV16_10LE32 }
0:00:06.974569108  4320  0x10cde60 DEBUG             capsfilter gstcapsfilter.c:297:gst_capsfilter_transform_caps:<capsfilter0> filter:    (NULL)
0:00:06.974616661  4320  0x10cde60 DEBUG             capsfilter gstcapsfilter.c:299:gst_capsfilter_transform_caps:<capsfilter0> caps filter:    video/x-raw, width=(int)1280, height=(int)720, framerate=(fraction)15/2, format=(string)YUY2
0:00:06.974651037  4320  0x10cde60 DEBUG             capsfilter gstcapsfilter.c:300:gst_capsfilter_transform_caps:<capsfilter0> intersect: EMPTY

Regards
Gaurav
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210818/5df1413c/attachment-0001.htm>