[External] AW: gstreamer videoconvert support gpu memory buffer
Matthew Waters
ystreet00 at gmail.com
Wed Aug 18 10:44:25 UTC 2021
Correct, doing so would require custom GStreamer modifications and this
should all really be your own specific element.
Cheers
-Matt
On 18/8/21 7:37 pm, Gaurav Vohra wrote:
>
> Thanks for reply.
>
> Is it not possible to do custom memory allocation in videoconvert as
> it can take care of pt.1 for my requirement.
>
> 1. format conversion from YUY2 to NV12
> 2. frame buffers from video/x-raw(system memory) to
> video/x-raw(memory:GBM)(gpu backed memory)
>
> And for pt2 after conversion modify out_frame to use gpu allocated
> memory ?
>
> static GstFlowReturn
>
> gst_video_convert_transform_frame (GstVideoFilter * filter,
>
> GstVideoFrame * in_frame, GstVideoFrame * out_frame)
>
> {
>
> The current issue is basically related to caps negotiation failing, as
> soon as I am modifying src_template to use memory:GBM instead of ANY
> negotiation is failing.
>
> +#define CSP_VIDEO_CAPS_SRC GST_VIDEO_CAPS_MAKE
> (GST_VIDEO_FORMATS_ALL) ";" \
>
> + GST_VIDEO_CAPS_MAKE_WITH_FEATURES ("memory:GBM",
> GST_VIDEO_FORMATS_ALL)
>
> +
>
> +//#define CSP_VIDEO_CAPS_SRC GST_VIDEO_CAPS_MAKE_WITH_FEATURES
> ("memory:GBM", GST_VIDEO_FORMATS_ALL)
>
> static GstStaticPadTemplate gst_video_convert_src_template =
>
> GST_STATIC_PAD_TEMPLATE ("src",
>
> GST_PAD_SRC,
>
> GST_PAD_ALWAYS,
>
> - GST_STATIC_CAPS (CSP_VIDEO_CAPS)
>
> + GST_STATIC_CAPS (CSP_VIDEO_CAPS_SRC)
>
> );
>
> ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
> Internal data stream error.
>
> Additional debug info:
>
> ../../../../gstreamer-1.14.4/libs/gst/base/gstbasesrc.c(3055):
> gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
>
> streaming stopped, reason not-negotiated (-4)
>
> Further debugging from logs suggest intersect of capsfilter input and
> the caps filter is coming as EMPTY
>
> 0:00:06.974521920 4320 0x10cde60 DEBUG capsfilter
> gstcapsfilter.c:296:gst_capsfilter_transform_caps:<capsfilter0>
> input: video/x-raw(memory:GBM), format=(string)NV12, width=(int)[
> 1, 32767 ], height=(int)[ 1, 32767 ], framerate=(fraction)[ 0/1,
> 2147483647/1 ]; video/x-raw(memory:GBM), width=(int)[ 1, 32767 ],
> height=(int)[ 1, 32767 ], framerate=(fraction)[ 0/1, 2147483647/1 ],
> format=(string){ I420, YV12, YUY2, UYVY, AYUV, RGBx, BGRx, xRGB, xBGR,
> RGBA, BGRA, ARGB, ABGR, RGB, BGR, Y41B, Y42B, YVYU, Y444, v210, v216,
> NV12, NV21, GRAY8, GRAY16_BE, GRAY16_LE, v308, RGB16, BGR16, RGB15,
> BGR15, UYVP, A420, RGB8P, YUV9, YVU9, IYU1, ARGB64, AYUV64, r210,
> I420_10BE, I420_10LE, I422_10BE, I422_10LE, Y444_10BE, Y444_10LE, GBR,
> GBR_10BE, GBR_10LE, NV16, NV24, NV12_64Z32, A420_10BE, A420_10LE,
> A422_10BE, A422_10LE, A444_10BE, A444_10LE, NV61, P010_10BE,
> P010_10LE, IYU2, VYUY, GBRA, GBRA_10BE, GBRA_10LE, GBR_12BE, GBR_12LE,
> GBRA_12BE, GBRA_12LE, I420_12BE, I420_12LE, I422_12BE, I422_12LE,
> Y444_12BE, Y444_12LE, GRAY10_LE32, NV12_10LE32, NV16_10LE32 }
>
> 0:00:06.974569108 4320 0x10cde60 DEBUG capsfilter
> gstcapsfilter.c:297:gst_capsfilter_transform_caps:<capsfilter0>
> filter: (NULL)
>
> 0:00:06.974616661 4320 0x10cde60 DEBUG capsfilter
> gstcapsfilter.c:299:gst_capsfilter_transform_caps:<capsfilter0> caps
> filter: video/x-raw, width=(int)1280, height=(int)720,
> framerate=(fraction)15/2, format=(string)YUY2
>
> 0:00:06.974651037 4320 0x10cde60 DEBUG capsfilter
> gstcapsfilter.c:300:gst_capsfilter_transform_caps:<capsfilter0>
> intersect: EMPTY
>
> Regards
>
> Gaurav
>
> *From:* Matthew Waters <ystreet00 at gmail.com>
> *Sent:* Wednesday, August 18, 2021 2:04 PM
> *To:* Discussion of the development of and with GStreamer
> <gstreamer-devel at lists.freedesktop.org>
> *Cc:* Gaurav Vohra <Gaurav.Vohra at einfochips.com>; Thornton, Keith
> <keith.thornton at zeiss.com>
> *Subject:* Re: [External] AW: gstreamer videoconvert support gpu
> memory buffer
>
> You should create your own element if you have custom memory requirements.
>
> Cheers
> -Matt
>
> On 18/8/21 6:17 pm, Gaurav Vohra via gstreamer-devel wrote:
>
> Hi,
>
> As I can see glcolorconvert expect sink and src as
> video/x-raw(memory:GLMemory), which is not same in my case as
> input for me is x-raw and o/p is x-raw(memory:GBM)
>
> In my case I am trying to access usb camera using /dev/video3 so
> input to videoconvert/or any other plugin would be
>
> video/x-raw, width=(int)1280, height=(int)720,
> framerate=(fraction)15/2, format=(string)YUY2
>
> and the aim is to convert using customized videoconvert/other
> element.
>
> video/x-raw(memory:GBM), width=(int)1280, height=(int)720,
> framerate=(fraction)15/2, format=(string)NV12
>
> the videoconvert customized is aimed to perform
>
> 1. format conversion from YUY2 to NV12
> 2. frame buffers from video/x-raw(system memory) to
> video/x-raw(memory:GBM)(gpu backed memory)
>
> Here GBM buffers are gpu backed memory which I am setting up in
> transform_frame of videoconvert.
>
> Above necessity is as qtimletflite requires sink input as NV12
> format and video/x-raw(memory:GBM)
>
> Playback Pipeline is as follows. Below videoconvert is customized
> version
>
> gst-launch-1.0 -v v4l2src device=/dev/video3 num-buffers=1 !
> video/x-raw,width=1280,height=720,framerate=15/2,format=YUY2 !
> videoconvert ! qtimletflite
> config=/data/misc/camera/mle_tflite.config
> model=/data/misc/camera/detect.tflite
> labels=/data/misc/camera/labelmap.txt postprocessing=detection !
> waylandsink
>
> Regards
>
> Gaurav
>
> *From:* gstreamer-devel
> <gstreamer-devel-bounces at lists.freedesktop.org>
> <mailto:gstreamer-devel-bounces at lists.freedesktop.org> *On Behalf
> Of *Thornton, Keith via gstreamer-devel
> *Sent:* Wednesday, August 18, 2021 1:14 PM
> *To:* Discussion of the development of and with GStreamer
> <gstreamer-devel at lists.freedesktop.org>
> <mailto:gstreamer-devel at lists.freedesktop.org>
> *Cc:* Thornton, Keith <keith.thornton at zeiss.com>
> <mailto:keith.thornton at zeiss.com>
> *Subject:* [External] AW: gstreamer videoconvert support gpu
> memory buffer
>
> *CAUTION:*This email originated from outside of the organization.
> This message might not be safe, use caution in opening it. If in
> doubt, do not open the attachment nor links in the message.
>
> Hi,
>
> does glcolorconvert not do what you need?
>
> Gruesse
>
> *Von:*gstreamer-devel
> <gstreamer-devel-bounces at lists.freedesktop.org
> <mailto:gstreamer-devel-bounces at lists.freedesktop.org>> *Im
> Auftrag von *Gaurav Vohra via gstreamer-devel
> *Gesendet:* Mittwoch, 18. August 2021 08:18
> *An:* gstreamer-devel at lists.freedesktop.org
> <mailto:gstreamer-devel at lists.freedesktop.org>
> *Cc:* Gaurav Vohra <Gaurav.Vohra at einfochips.com
> <mailto:Gaurav.Vohra at einfochips.com>>
> *Betreff:* gstreamer videoconvert support gpu memory buffer
>
> Hello,
>
> I am working to customize gstreamer plugin to support
>
> working to modify gstreamer plugin - videoconvert get obtain
> output data allocated as gpu memory allocated buffers.
>
> As part of first step have modified the plugin to update the src
> template to include the gpu memory caps.
>
> Allocation for gpu memory is done is transform_frame, but before
> that being called stream error and negotition is failing
>
> Commented part
>
> // #define CSP_VIDEO_CAPS GST_VIDEO_CAPS_MAKE
> (GST_VIDEO_FORMATS_ALL) ";" \
>
> GST_VIDEO_CAPS_MAKE_WITH_FEATURES ("ANY", GST_VIDEO_FORMATS_ALL)
>
> Modified part
>
> #define CSP_VIDEO_CAPS_SRC GST_VIDEO_CAPS_MAKE
> (GST_VIDEO_FORMATS_ALL) ";" \
>
> GST_VIDEO_CAPS_MAKE_WITH_FEATURES ("memory:GBM",
> GST_VIDEO_FORMATS_ALL)
>
> //#define CSP_VIDEO_CAPS_SRC GST_VIDEO_CAPS_MAKE_WITH_FEATURES
> ("memory:GBM", GST_VIDEO_FORMATS_ALL)
>
> static GstStaticPadTemplate gst_video_convert_src_template =
>
> GST_STATIC_PAD_TEMPLATE ("src",
>
> GST_PAD_SRC,
>
> GST_PAD_ALWAYS,
>
> GST_STATIC_CAPS (CSP_VIDEO_CAPS_SRC)
>
> );
>
> While trying to run the playback pipeline getting errors related
> to negotitation and stream error like below
>
> ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
> Internal data stream error.
>
> Additional debug info:
>
> ../../../../gstreamer-1.14.4/libs/gst/base/gstbasesrc.c(3055):
> gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
>
> streaming stopped, reason not-negotiated (-4)
>
> Further debugging from logs suggest intersect of capsfilter input
> and the caps filter is coming as EMPTY
>
> 0:00:06.974521920 4320 0x10cde60 DEBUG capsfilter
> gstcapsfilter.c:296:gst_capsfilter_transform_caps:<capsfilter0>
> input: video/x-raw(memory:GBM), format=(string)NV12,
> width=(int)[ 1, 32767 ], height=(int)[ 1, 32767 ],
> framerate=(fraction)[ 0/1, 2147483647/1 ];
> video/x-raw(memory:GBM), width=(int)[ 1, 32767 ], height=(int)[ 1,
> 32767 ], framerate=(fraction)[ 0/1, 2147483647/1 ],
> format=(string){ I420, YV12, YUY2, UYVY, AYUV, RGBx, BGRx, xRGB,
> xBGR, RGBA, BGRA, ARGB, ABGR, RGB, BGR, Y41B, Y42B, YVYU, Y444,
> v210, v216, NV12, NV21, GRAY8, GRAY16_BE, GRAY16_LE, v308, RGB16,
> BGR16, RGB15, BGR15, UYVP, A420, RGB8P, YUV9, YVU9, IYU1, ARGB64,
> AYUV64, r210, I420_10BE, I420_10LE, I422_10BE, I422_10LE,
> Y444_10BE, Y444_10LE, GBR, GBR_10BE, GBR_10LE, NV16, NV24,
> NV12_64Z32, A420_10BE, A420_10LE, A422_10BE, A422_10LE, A444_10BE,
> A444_10LE, NV61, P010_10BE, P010_10LE, IYU2, VYUY, GBRA,
> GBRA_10BE, GBRA_10LE, GBR_12BE, GBR_12LE, GBRA_12BE, GBRA_12LE,
> I420_12BE, I420_12LE, I422_12BE, I422_12LE, Y444_12BE, Y444_12LE,
> GRAY10_LE32, NV12_10LE32, NV16_10LE32 }
>
> 0:00:06.974569108 4320 0x10cde60 DEBUG capsfilter
> gstcapsfilter.c:297:gst_capsfilter_transform_caps:<capsfilter0>
> filter: (NULL)
>
> 0:00:06.974616661 4320 0x10cde60 DEBUG capsfilter
> gstcapsfilter.c:299:gst_capsfilter_transform_caps:<capsfilter0>
> caps filter: video/x-raw, width=(int)1280, height=(int)720,
> framerate=(fraction)15/2, format=(string)YUY2
>
> 0:00:06.974651037 4320 0x10cde60 DEBUG capsfilter
> gstcapsfilter.c:300:gst_capsfilter_transform_caps:<capsfilter0>
> intersect: EMPTY
>
> Regards
>
> Gaurav
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210818/1efc04e4/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210818/1efc04e4/attachment-0001.sig>
More information about the gstreamer-devel
mailing list