<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv="Content-Type" content="text/html; charset=us-ascii"> <meta name="Generator" content="Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang="EN-US" link="#0563C1" vlink="purple" style="word-wrap:break-word"> <div class="WordSection1"> Yes, understand the my requirement would require gstreamer plugin modifications or creating new element, so I am giving a try to incorporate changes as of now to videoconvert.<o:p></o:p> <o:p> </o:p> Any pointers about the error part for negotiation failure mentioned earlier, as I am just modifying GstStaticPadTemplate and the pipeline is in PLAYING state before the negotiation fails.<o:p></o:p> <o:p> </o:p> OR<o:p></o:p> <o:p> </o:p> Any pointers related to implementation of my requirements if any specific(e.g baseclass to be used etc.)<o:p></o:p> <o:p> </o:p> Thanks<o:p></o:p> Gaurav<o:p></o:p> <div> <div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in"> From: Matthew Waters <ystreet00@gmail.com> Sent: Wednesday, August 18, 2021 4:14 PM To: Gaurav Vohra <Gaurav.Vohra@einfochips.com>; Discussion of the development of and with GStreamer <gstreamer-devel@lists.freedesktop.org> Cc: Thornton, Keith <keith.thornton@zeiss.com> Subject: Re: [External] AW: gstreamer videoconvert support gpu memory buffer<o:p></o:p> </div> </div> <o:p> </o:p> Correct, doing so would require custom GStreamer modifications and this should all really be your own specific element. Cheers -Matt<o:p></o:p> <div> On 18/8/21 7:37 pm, Gaurav Vohra wrote:<o:p></o:p> </div> <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"> Thanks for reply.<o:p></o:p> <o:p></o:p> Is it not possible to do custom memory allocation in videoconvert as it can take care of pt.1 for my requirement.<o:p></o:p> <o:p></o:p> <ol style="margin-top:0in" start="1" type="1"> <li class="MsoListParagraph" style="margin-left:0in;mso-list:l2 level1 lfo3">format conversion from YUY2 to NV12<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l2 level1 lfo3">frame buffers from video/x-raw(system memory) to video/x-raw(memory:GBM)(gpu backed memory)<o:p></o:p></li></ol> <o:p></o:p> And for pt2 after conversion modify out_frame to use gpu allocated memory ?<o:p></o:p> <o:p></o:p> static GstFlowReturn<o:p></o:p> gst_video_convert_transform_frame (GstVideoFilter * filter,<o:p></o:p> GstVideoFrame * in_frame, GstVideoFrame * out_frame)<o:p></o:p> {<o:p></o:p> <o:p></o:p> The current issue is basically related to caps negotiation failing, as soon as I am modifying src_template to use memory:GBM instead of ANY negotiation is failing.<o:p></o:p> <o:p></o:p> +#define CSP_VIDEO_CAPS_SRC GST_VIDEO_CAPS_MAKE (GST_VIDEO_FORMATS_ALL) ";" \<o:p></o:p> + GST_VIDEO_CAPS_MAKE_WITH_FEATURES ("memory:GBM", GST_VIDEO_FORMATS_ALL)<o:p></o:p> +<o:p></o:p> +//#define CSP_VIDEO_CAPS_SRC GST_VIDEO_CAPS_MAKE_WITH_FEATURES ("memory:GBM", GST_VIDEO_FORMATS_ALL)<o:p></o:p> static GstStaticPadTemplate gst_video_convert_src_template =<o:p></o:p> GST_STATIC_PAD_TEMPLATE ("src",<o:p></o:p> GST_PAD_SRC,<o:p></o:p> GST_PAD_ALWAYS,<o:p></o:p> - GST_STATIC_CAPS (CSP_VIDEO_CAPS)<o:p></o:p> + GST_STATIC_CAPS (CSP_VIDEO_CAPS_SRC)<o:p></o:p> );<o:p></o:p> <o:p></o:p> ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.<o:p></o:p> Additional debug info:<o:p></o:p> ../../../../gstreamer-1.14.4/libs/gst/base/gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:<o:p></o:p> streaming stopped, reason not-negotiated (-4)<o:p></o:p> <o:p></o:p> Further debugging from logs suggest intersect of capsfilter input and the caps filter is coming as EMPTY <o:p></o:p> <o:p></o:p> 0:00:06.974521920 4320 0x10cde60 DEBUG capsfilter gstcapsfilter.c:296:gst_capsfilter_transform_caps:<capsfilter0> input: video/x-raw(memory:GBM), format=(string)NV12, width=(int)[ 1, 32767 ], height=(int)[ 1, 32767 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw(memory:GBM), width=(int)[ 1, 32767 ], height=(int)[ 1, 32767 ], framerate=(fraction)[ 0/1, 2147483647/1 ], format=(string){ I420, YV12, YUY2, UYVY, AYUV, RGBx, BGRx, xRGB, xBGR, RGBA, BGRA, ARGB, ABGR, RGB, BGR, Y41B, Y42B, YVYU, Y444, v210, v216, NV12, NV21, GRAY8, GRAY16_BE, GRAY16_LE, v308, RGB16, BGR16, RGB15, BGR15, UYVP, A420, RGB8P, YUV9, YVU9, IYU1, ARGB64, AYUV64, r210, I420_10BE, I420_10LE, I422_10BE, I422_10LE, Y444_10BE, Y444_10LE, GBR, GBR_10BE, GBR_10LE, NV16, NV24, NV12_64Z32, A420_10BE, A420_10LE, A422_10BE, A422_10LE, A444_10BE, A444_10LE, NV61, P010_10BE, P010_10LE, IYU2, VYUY, GBRA, GBRA_10BE, GBRA_10LE, GBR_12BE, GBR_12LE, GBRA_12BE, GBRA_12LE, I420_12BE, I420_12LE, I422_12BE, I422_12LE, Y444_12BE, Y444_12LE, GRAY10_LE32, NV12_10LE32, NV16_10LE32 }<o:p></o:p> 0:00:06.974569108 4320 0x10cde60 DEBUG capsfilter gstcapsfilter.c:297:gst_capsfilter_transform_caps:<capsfilter0> filter: (NULL)<o:p></o:p> 0:00:06.974616661 4320 0x10cde60 DEBUG capsfilter gstcapsfilter.c:299:gst_capsfilter_transform_caps:<capsfilter0> caps filter: video/x-raw, width=(int)1280, height=(int)720, framerate=(fraction)15/2, format=(string)YUY2<o:p></o:p> 0:00:06.974651037 4320 0x10cde60 DEBUG capsfilter gstcapsfilter.c:300:gst_capsfilter_transform_caps:<capsfilter0> intersect: EMPTY<o:p></o:p> <o:p></o:p> Regards<o:p></o:p> Gaurav<o:p></o:p> <div> <div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in"> From: Matthew Waters <a href="mailto:ystreet00@gmail.com"> <ystreet00@gmail.com></a> Sent: Wednesday, August 18, 2021 2:04 PM To: Discussion of the development of and with GStreamer <a href="mailto:gstreamer-devel@lists.freedesktop.org"> <gstreamer-devel@lists.freedesktop.org></a> Cc: Gaurav Vohra <a href="mailto:Gaurav.Vohra@einfochips.com"><Gaurav.Vohra@einfochips.com></a>; Thornton, Keith <a href="mailto:keith.thornton@zeiss.com"><keith.thornton@zeiss.com></a> Subject: Re: [External] AW: gstreamer videoconvert support gpu memory buffer<o:p></o:p> </div> </div> <o:p></o:p> You should create your own element if you have custom memory requirements. Cheers -Matt<o:p></o:p> <div> On 18/8/21 6:17 pm, Gaurav Vohra via gstreamer-devel wrote:<o:p></o:p> </div> <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"> Hi,<o:p></o:p> <o:p></o:p> As I can see glcolorconvert expect sink and src as video/x-raw(memory:GLMemory), which is not same in my case as input for me is x-raw and o/p is x-raw(memory:GBM)<o:p></o:p> <o:p></o:p> In my case I am trying to access usb camera using /dev/video3 so input to videoconvert/or any other plugin would be <o:p></o:p> <o:p></o:p> video/x-raw, width=(int)1280, height=(int)720, framerate=(fraction)15/2, format=(string)YUY2<o:p></o:p> <o:p></o:p> and the aim is to convert using customized videoconvert/other element. <o:p></o:p> <o:p></o:p> video/x-raw(memory:GBM), width=(int)1280, height=(int)720, framerate=(fraction)15/2, format=(string)NV12<o:p></o:p> <o:p></o:p> the videoconvert customized is aimed to perform<o:p></o:p> <o:p></o:p> <ol style="margin-top:0in" start="1" type="1"> <li class="MsoListParagraph" style="margin-left:0in;mso-list:l4 level1 lfo6">format conversion from YUY2 to NV12<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l4 level1 lfo6">frame buffers from video/x-raw(system memory) to video/x-raw(memory:GBM)(gpu backed memory)<o:p></o:p></li></ol> <o:p></o:p> Here GBM buffers are gpu backed memory which I am setting up in transform_frame of videoconvert.<o:p></o:p> <o:p></o:p> Above necessity is as qtimletflite requires sink input as NV12 format and video/x-raw(memory:GBM)<o:p></o:p> <o:p></o:p> Playback Pipeline is as follows. Below videoconvert is customized version<o:p></o:p> <o:p></o:p> gst-launch-1.0 -v v4l2src device=/dev/video3 num-buffers=1 ! video/x-raw,width=1280,height=720,framerate=15/2,format=YUY2 ! videoconvert ! qtimletflite config=/data/misc/camera/mle_tflite.config model=/data/misc/camera/detect.tflite labels=/data/misc/camera/labelmap.txt postprocessing=detection ! waylandsink<o:p></o:p> <o:p></o:p> Regards<o:p></o:p> Gaurav<o:p></o:p> <div> <div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in"> From: gstreamer-devel <a href="mailto:gstreamer-devel-bounces@lists.freedesktop.org"> <gstreamer-devel-bounces@lists.freedesktop.org></a> On Behalf Of Thornton, Keith via gstreamer-devel Sent: Wednesday, August 18, 2021 1:14 PM To: Discussion of the development of and with GStreamer <a href="mailto:gstreamer-devel@lists.freedesktop.org"> <gstreamer-devel@lists.freedesktop.org></a> Cc: Thornton, Keith <a href="mailto:keith.thornton@zeiss.com"><keith.thornton@zeiss.com></a> Subject: [External] AW: gstreamer videoconvert support gpu memory buffer<o:p></o:p> </div> </div> <o:p></o:p> <o:p></o:p> <div style="border:solid #9C6500 1.0pt;padding:2.0pt 2.0pt 2.0pt 2.0pt"> CAUTION: This email originated from outside of the organization. This message might not be safe, use caution in opening it. If in doubt, do not open the attachment nor links in the message.<o:p></o:p> </div> <o:p></o:p> <div> Hi,<o:p></o:p> does glcolorconvert not do what you need?<o:p></o:p> Gruesse<o:p></o:p> <o:p></o:p> <div> <div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in"> Von: gstreamer-devel <<a href="mailto:gstreamer-devel-bounces@lists.freedesktop.org">gstreamer-devel-bounces@lists.freedesktop.org</a>> Im Auftrag von Gaurav Vohra via gstreamer-devel Gesendet: Mittwoch, 18. August 2021 08:18 An: <a href="mailto:gstreamer-devel@lists.freedesktop.org">gstreamer-devel@lists.freedesktop.org</a> Cc: Gaurav Vohra <<a href="mailto:Gaurav.Vohra@einfochips.com">Gaurav.Vohra@einfochips.com</a>> Betreff: gstreamer videoconvert support gpu memory buffer<o:p></o:p> </div> </div> <o:p></o:p> Hello,<o:p></o:p> <o:p></o:p> I am working to customize gstreamer plugin to support <o:p></o:p> <o:p></o:p> working to modify gstreamer plugin - videoconvert get obtain output data allocated as gpu memory allocated buffers.<o:p></o:p> As part of first step have modified the plugin to update the src template to include the gpu memory caps.<o:p></o:p> Allocation for gpu memory is done is transform_frame, but before that being called stream error and negotition is failing<o:p></o:p> Commented part<o:p></o:p> // #define CSP_VIDEO_CAPS GST_VIDEO_CAPS_MAKE (GST_VIDEO_FORMATS_ALL) ";" \<o:p></o:p> GST_VIDEO_CAPS_MAKE_WITH_FEATURES ("ANY", GST_VIDEO_FORMATS_ALL)<o:p></o:p> Modified part<o:p></o:p> #define CSP_VIDEO_CAPS_SRC GST_VIDEO_CAPS_MAKE (GST_VIDEO_FORMATS_ALL) ";" \<o:p></o:p> GST_VIDEO_CAPS_MAKE_WITH_FEATURES ("memory:GBM", GST_VIDEO_FORMATS_ALL)<o:p></o:p> <o:p></o:p> //#define CSP_VIDEO_CAPS_SRC GST_VIDEO_CAPS_MAKE_WITH_FEATURES ("memory:GBM", GST_VIDEO_FORMATS_ALL)<o:p></o:p> static GstStaticPadTemplate gst_video_convert_src_template =<o:p></o:p> GST_STATIC_PAD_TEMPLATE ("src",<o:p></o:p> GST_PAD_SRC,<o:p></o:p> GST_PAD_ALWAYS,<o:p></o:p> GST_STATIC_CAPS (CSP_VIDEO_CAPS_SRC)<o:p></o:p> );<o:p></o:p> While trying to run the playback pipeline getting errors related to negotitation and stream error like below<o:p></o:p> ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.<o:p></o:p> Additional debug info:<o:p></o:p> ../../../../gstreamer-1.14.4/libs/gst/base/gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:<o:p></o:p> streaming stopped, reason not-negotiated (-4)<o:p></o:p> <o:p></o:p> Further debugging from logs suggest intersect of capsfilter input and the caps filter is coming as EMPTY <o:p></o:p> <o:p></o:p> 0:00:06.974521920 4320 0x10cde60 DEBUG capsfilter gstcapsfilter.c:296:gst_capsfilter_transform_caps:<capsfilter0> input: video/x-raw(memory:GBM), format=(string)NV12, width=(int)[ 1, 32767 ], height=(int)[ 1, 32767 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw(memory:GBM), width=(int)[ 1, 32767 ], height=(int)[ 1, 32767 ], framerate=(fraction)[ 0/1, 2147483647/1 ], format=(string){ I420, YV12, YUY2, UYVY, AYUV, RGBx, BGRx, xRGB, xBGR, RGBA, BGRA, ARGB, ABGR, RGB, BGR, Y41B, Y42B, YVYU, Y444, v210, v216, NV12, NV21, GRAY8, GRAY16_BE, GRAY16_LE, v308, RGB16, BGR16, RGB15, BGR15, UYVP, A420, RGB8P, YUV9, YVU9, IYU1, ARGB64, AYUV64, r210, I420_10BE, I420_10LE, I422_10BE, I422_10LE, Y444_10BE, Y444_10LE, GBR, GBR_10BE, GBR_10LE, NV16, NV24, NV12_64Z32, A420_10BE, A420_10LE, A422_10BE, A422_10LE, A444_10BE, A444_10LE, NV61, P010_10BE, P010_10LE, IYU2, VYUY, GBRA, GBRA_10BE, GBRA_10LE, GBR_12BE, GBR_12LE, GBRA_12BE, GBRA_12LE, I420_12BE, I420_12LE, I422_12BE, I422_12LE, Y444_12BE, Y444_12LE, GRAY10_LE32, NV12_10LE32, NV16_10LE32 }<o:p></o:p> 0:00:06.974569108 4320 0x10cde60 DEBUG capsfilter gstcapsfilter.c:297:gst_capsfilter_transform_caps:<capsfilter0> filter: (NULL)<o:p></o:p> 0:00:06.974616661 4320 0x10cde60 DEBUG capsfilter gstcapsfilter.c:299:gst_capsfilter_transform_caps:<capsfilter0> caps filter: video/x-raw, width=(int)1280, height=(int)720, framerate=(fraction)15/2, format=(string)YUY2<o:p></o:p> 0:00:06.974651037 4320 0x10cde60 DEBUG capsfilter gstcapsfilter.c:300:gst_capsfilter_transform_caps:<capsfilter0> intersect: EMPTY<o:p></o:p> <o:p></o:p> Regards<o:p></o:p> Gaurav<o:p></o:p> </div> </blockquote> <o:p></o:p> </blockquote> <o:p> </o:p> </div> </body> </html>