[External] AW: gstreamer videoconvert support gpu memory buffer

Wed Aug 18 08:33:32 UTC 2021

You should create your own element if you have custom memory requirements.

Cheers
-Matt

On 18/8/21 6:17 pm, Gaurav Vohra via gstreamer-devel wrote:
>
> Hi,
>
> As I can see glcolorconvert expect sink and src as 
> video/x-raw(memory:GLMemory), which is not same in my case as input 
> for me is x-raw and o/p is x-raw(memory:GBM)
>
> In my case I am trying to access usb camera using /dev/video3 so input 
> to videoconvert/or any other plugin would be
>
> video/x-raw, width=(int)1280, height=(int)720, 
> framerate=(fraction)15/2, format=(string)YUY2
>
> and the aim is to convert using customized videoconvert/other element.
>
> video/x-raw(memory:GBM), width=(int)1280, height=(int)720, 
> framerate=(fraction)15/2, format=(string)NV12
>
> the videoconvert customized is aimed to perform
>
>  1. format conversion from YUY2 to NV12
>  2. frame buffers from video/x-raw(system memory) to
>     video/x-raw(memory:GBM)(gpu backed memory)
>
> Here GBM buffers are gpu backed memory which I am setting up in 
> transform_frame of videoconvert.
>
> Above necessity is as qtimletflite requires sink input as NV12 format 
> and video/x-raw(memory:GBM)
>
> Playback Pipeline is as follows. Below videoconvert is customized version
>
> gst-launch-1.0 -v v4l2src device=/dev/video3 num-buffers=1 ! 
> video/x-raw,width=1280,height=720,framerate=15/2,format=YUY2 ! 
> videoconvert ! qtimletflite config=/data/misc/camera/mle_tflite.config 
> model=/data/misc/camera/detect.tflite 
> labels=/data/misc/camera/labelmap.txt postprocessing=detection ! 
> waylandsink
>
> Regards
>
> Gaurav
>
> *From:* gstreamer-devel 
> <gstreamer-devel-bounces at lists.freedesktop.org> *On Behalf Of 
> *Thornton, Keith via gstreamer-devel
> *Sent:* Wednesday, August 18, 2021 1:14 PM
> *To:* Discussion of the development of and with GStreamer 
> <gstreamer-devel at lists.freedesktop.org>
> *Cc:* Thornton, Keith <keith.thornton at zeiss.com>
> *Subject:* [External] AW: gstreamer videoconvert support gpu memory buffer
>
> *CAUTION:*This email originated from outside of the organization. This 
> message might not be safe, use caution in opening it. If in doubt, do 
> not open the attachment nor links in the message.
>
> Hi,
>
> does glcolorconvert not do what you need?
>
> Gruesse
>
> *Von:*gstreamer-devel <gstreamer-devel-bounces at lists.freedesktop.org 
> <mailto:gstreamer-devel-bounces at lists.freedesktop.org>> *Im Auftrag 
> von *Gaurav Vohra via gstreamer-devel
> *Gesendet:* Mittwoch, 18. August 2021 08:18
> *An:* gstreamer-devel at lists.freedesktop.org 
> <mailto:gstreamer-devel at lists.freedesktop.org>
> *Cc:* Gaurav Vohra <Gaurav.Vohra at einfochips.com 
> <mailto:Gaurav.Vohra at einfochips.com>>
> *Betreff:* gstreamer videoconvert support gpu memory buffer
>
> Hello,
>
> I am working to customize gstreamer plugin to support
>
> working to modify gstreamer plugin - videoconvert get obtain output 
> data allocated as gpu memory allocated buffers.
>
> As part of first step have modified the plugin to update the src 
> template to include the gpu memory caps.
>
> Allocation for gpu memory is done is transform_frame, but before that 
> being called stream error and negotition is failing
>
> Commented part
>
> // #define CSP_VIDEO_CAPS GST_VIDEO_CAPS_MAKE (GST_VIDEO_FORMATS_ALL) 
> ";" \
>
> GST_VIDEO_CAPS_MAKE_WITH_FEATURES ("ANY", GST_VIDEO_FORMATS_ALL)
>
> Modified part
>
> #define CSP_VIDEO_CAPS_SRC GST_VIDEO_CAPS_MAKE (GST_VIDEO_FORMATS_ALL) 
> ";" \
>
> GST_VIDEO_CAPS_MAKE_WITH_FEATURES ("memory:GBM", GST_VIDEO_FORMATS_ALL)
>
> //#define CSP_VIDEO_CAPS_SRC GST_VIDEO_CAPS_MAKE_WITH_FEATURES 
> ("memory:GBM", GST_VIDEO_FORMATS_ALL)
>
> static GstStaticPadTemplate gst_video_convert_src_template =
>
> GST_STATIC_PAD_TEMPLATE ("src",
>
> GST_PAD_SRC,
>
> GST_PAD_ALWAYS,
>
> GST_STATIC_CAPS (CSP_VIDEO_CAPS_SRC)
>
>     );
>
> While trying to run the playback pipeline getting errors related to 
> negotitation and stream error like below
>
> ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: 
> Internal data stream error.
>
> Additional debug info:
>
> ../../../../gstreamer-1.14.4/libs/gst/base/gstbasesrc.c(3055): 
> gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
>
> streaming stopped, reason not-negotiated (-4)
>
> Further debugging from logs suggest intersect of capsfilter input and 
> the caps filter is coming as EMPTY
>
> 0:00:06.974521920  4320  0x10cde60 DEBUG             capsfilter 
> gstcapsfilter.c:296:gst_capsfilter_transform_caps:<capsfilter0> 
> input:     video/x-raw(memory:GBM), format=(string)NV12, width=(int)[ 
> 1, 32767 ], height=(int)[ 1, 32767 ], framerate=(fraction)[ 0/1, 
> 2147483647/1 ]; video/x-raw(memory:GBM), width=(int)[ 1, 32767 ], 
> height=(int)[ 1, 32767 ], framerate=(fraction)[ 0/1, 2147483647/1 ], 
> format=(string){ I420, YV12, YUY2, UYVY, AYUV, RGBx, BGRx, xRGB, xBGR, 
> RGBA, BGRA, ARGB, ABGR, RGB, BGR, Y41B, Y42B, YVYU, Y444, v210, v216, 
> NV12, NV21, GRAY8, GRAY16_BE, GRAY16_LE, v308, RGB16, BGR16, RGB15, 
> BGR15, UYVP, A420, RGB8P, YUV9, YVU9, IYU1, ARGB64, AYUV64, r210, 
> I420_10BE, I420_10LE, I422_10BE, I422_10LE, Y444_10BE, Y444_10LE, GBR, 
> GBR_10BE, GBR_10LE, NV16, NV24, NV12_64Z32, A420_10BE, A420_10LE, 
> A422_10BE, A422_10LE, A444_10BE, A444_10LE, NV61, P010_10BE, 
> P010_10LE, IYU2, VYUY, GBRA, GBRA_10BE, GBRA_10LE, GBR_12BE, GBR_12LE, 
> GBRA_12BE, GBRA_12LE, I420_12BE, I420_12LE, I422_12BE, I422_12LE, 
> Y444_12BE, Y444_12LE, GRAY10_LE32, NV12_10LE32, NV16_10LE32 }
>
> 0:00:06.974569108  4320  0x10cde60 DEBUG             capsfilter 
> gstcapsfilter.c:297:gst_capsfilter_transform_caps:<capsfilter0> 
> filter:    (NULL)
>
> 0:00:06.974616661  4320  0x10cde60 DEBUG             capsfilter 
> gstcapsfilter.c:299:gst_capsfilter_transform_caps:<capsfilter0> caps 
> filter:    video/x-raw, width=(int)1280, height=(int)720, 
> framerate=(fraction)15/2, format=(string)YUY2
>
> 0:00:06.974651037  4320  0x10cde60 DEBUG             capsfilter 
> gstcapsfilter.c:300:gst_capsfilter_transform_caps:<capsfilter0> 
> intersect: EMPTY
>
> Regards
>
> Gaurav
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210818/1d09e845/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210818/1d09e845/attachment-0001.sig>