Conversion of cv::Mat to GstBuffer

[philippe renon]

Note that there a quite a few existing opencv plugins in gst-plugins-bad.
Most of them extend gstopencvvideofilter which provides the logic for converting a cv::Mat to a GstBugger (and the other way around).
See https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/blob/master/gst-libs/gst/opencv/gstopencvvideofilter.cpp.


 

    Le Mercredi 13 mars 2019 19h18, Michael Gruner <michael.gruner at ridgerun.com> a écrit :
 

 It looks like img2, being a local variable, ends  its life when “cb_need_data” ends. That leaves the pointer in “buffer” invalid. For this one you’re better off with the copy. 
Another option is to alloc img2 on the heap, and use gst_buffer_new_wrapped_full to carry img2 along with buffer and free then together.

Michaelwww.ridgerun.com
On Mar 13, 2019, at 10:51 AM, Saurabh Bora <saurabh9bora at gmail.com> wrote:


Hi Michael, 
Thanks! for your input. I tried what you suggested as below:
//---------------------------------------------------------------#include <gst/gst.h>#include <opencv2/opencv.hpp>#include <opencv2/highgui.hpp>#include <opencv2/imgproc/imgproc.hpp>
static GMainLoop *loop;
static voidcb_need_data(GstElement *appsrc, guint       unused_size, gpointer    user_data){ static GstClockTime timestamp = 0; GstBuffer *buffer; guint size, depth, height, width, step, channels; GstFlowReturn ret; guchar *data1; GstMapInfo map;
 cv::Mat img2; cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg"); cv::cvtColor(img, img2, cv::COLOR_BGR2RGB);
 height = img.size().height; width = img.size().width; //step = img.widthStep; channels = img.channels(); //depth = img->depth; data1 = (guchar *)img.data; size = height * width * channels * sizeof(guchar);
 std::cout << "cb_need_data called" << std::endl;
 buffer = gst_buffer_new_wrapped(img2.data, img2.total() * img2.elemSize());
 GstMapInfo info; gst_buffer_map(buffer, &info, GST_MAP_READ); cv::Mat mat(height, width, CV_8UC3, info.data); cv::imshow("Output", mat); //gst_buffer_unmap(buffer, &info);
 GST_BUFFER_PTS(buffer) = timestamp; GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);
 timestamp += GST_BUFFER_DURATION(buffer);
 g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret); //gst_buffer_unref(buffer);
 if (ret != GST_FLOW_OK) { /* something wrong, stop pushing */ g_main_loop_quit(loop); }}
gintmain(gint   argc, gchar *argv[]){ GstElement *pipeline, *appsrc, *conv, *videosink;
 /* init GStreamer */ gst_init(&argc, &argv); loop = g_main_loop_new(NULL, FALSE);
 /* setup pipeline */ pipeline = gst_pipeline_new("pipeline"); appsrc = gst_element_factory_make("appsrc", "source"); conv = gst_element_factory_make("videoconvert", "conv"); videosink = gst_element_factory_make("autovideosink", "videosink");
 /* setup */ g_object_set(G_OBJECT(appsrc), "caps", gst_caps_new_simple("video/x-raw", "format", G_TYPE_STRING, "RGB", "width", G_TYPE_INT, 134, "height", G_TYPE_INT, 134, "framerate", GST_TYPE_FRACTION, 1, 1, NULL), NULL); gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL); gst_element_link_many(appsrc, conv, videosink, NULL); //g_object_set (videosink, "device", "/dev/video0", NULL); /* setup appsrc */ g_object_set(G_OBJECT(appsrc), "stream-type", 0, "format", GST_FORMAT_TIME, NULL); g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);
 /* play */ gst_element_set_state(pipeline, GST_STATE_PLAYING); g_main_loop_run(loop);
 if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE) { g_printerr("Unable to set the pipeline to the playing state.\n"); }
 // Free resources gst_object_unref(GST_OBJECT(pipeline)); g_main_loop_unref(loop);
 return 0;}//----------------------------------------------------------------------------------------------
It still does not work as expected. It would be a great help if you could point out where I am going wrong.Thanks!!
On Wed, Mar 13, 2019 at 7:40 PM Michael Gruner <michael.gruner at ridgerun.com> wrote:

Hi Saurabh
Yo may convert from a cv::Mat to a GstBuffer without a memory copy by using gst_buffer_new_wrapped:
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstBuffer.html#gst-buffer-new-wrapped
Something like: 
buffer = gst_buffer_new_wrapped (mat.data, mat.total()*mat.elemSize());
In order to make a cv::Mat from a GstBuffer you first need to take the data out of the buffer. 
GstMapInfo info;gst_buffer_map (buffer, &info, GST_MAP_READ); // or writemat = cv::Mat(height, width, CV_8C3, map.data); // change your format accordingly...gst_buffer_unmap (buffer, &info);
Hope it helps.
Michaelwww.ridgerun.com
On Mar 13, 2019, at 7:15 AM, Saurabh Bora <saurabh9bora at gmail.com> wrote:


Hi Experts!!, 
I am trying to read opencv images locally and push them into gstreamer pipeline. I am using appsrc element with "need-data" signal attached to call-back function.Please look at the code below. 
//------------------------------------------------------------------------------------------------------#include <gst/gst.h>#include <opencv2/opencv.hpp>#include <opencv2/highgui.hpp>
static GMainLoop *loop;
static voidcb_need_data(GstElement *appsrc, guint       unused_size, gpointer    user_data){ static GstClockTime timestamp = 0; GstBuffer *buffer; guint size, depth, height, width, step, channels; GstFlowReturn ret; guchar *data1; GstMapInfo map;
 cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg"); height = img.size().height; width = img.size().width; //step = img.widthStep; channels = img.channels(); //depth = img->depth; data1 = (guchar *)img.data; size = height * width* channels;
 // Copy cv::Mat to GstBuffer buffer = gst_buffer_new_allocate(NULL, size, NULL); gst_buffer_map(buffer, &map, GST_MAP_WRITE); memcpy((guchar *)map.data, data1, gst_buffer_get_size(buffer));
 //Convert GstBuffer back to cv::Mat and write it, to check if data was correctly copied. //This is where I get to know that data is not correct. cv::Mat img2(cv::Size(134, 134), CV_8UC3, (char*)(buffer)); cv::imwrite("C:/Users/212730892.LOGON/Desktop/dummy1.jpg", img2);
 GST_BUFFER_PTS(buffer) = timestamp; GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);
 timestamp += GST_BUFFER_DURATION(buffer);
 g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret); //gst_buffer_unref(buffer);
 if (ret != GST_FLOW_OK) { /* something wrong, stop pushing */ g_main_loop_quit(loop); }}
gintmain(gint   argc, gchar *argv[]){ GstElement *pipeline, *appsrc, *conv, *videosink;
 /* init GStreamer */ gst_init(&argc, &argv); loop = g_main_loop_new(NULL, FALSE);
 /* setup pipeline */ pipeline = gst_pipeline_new("pipeline"); appsrc = gst_element_factory_make("appsrc", "source"); conv = gst_element_factory_make("videoconvert", "conv"); videosink = gst_element_factory_make("autovideosink", "videosink");
 /* setup */ g_object_set(G_OBJECT(appsrc), "caps", gst_caps_new_simple("video/x-raw", "format", G_TYPE_STRING, "RGB", "width", G_TYPE_INT, 134, "height", G_TYPE_INT, 134, "framerate", GST_TYPE_FRACTION, 1, 1, NULL), NULL); gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL); gst_element_link_many(appsrc, conv, videosink, NULL); //g_object_set (videosink, "device", "/dev/video0", NULL); /* setup appsrc */ g_object_set(G_OBJECT(appsrc), "stream-type", 0, "format", GST_FORMAT_TIME, NULL); g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);
 /* play */ gst_element_set_state(pipeline, GST_STATE_PLAYING); g_main_loop_run(loop);
 if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE) { g_printerr("Unable to set the pipeline to the playing state.\n"); }
 // Free resources gst_object_unref(GST_OBJECT(pipeline)); g_main_loop_unref(loop);
 return 0;}//-------------------------------------------------------------------------------------------------------------------------------------
In cb_need_data, the part where copying I am copying to GstBuffer seems to be wrong and it does not work as expected. Can anyone, please help me understand, how can I copy the image data perfectly and pass it downstream to other elements without losing any data?
-- 
Thanks and Regards,Saurabh Bora

PH NO : 7038166900EMAIL : saurabh9bora at gmail.com             saurabh9bora at outlook.com

_______________________________________________
gstreamer-devel mailing list
gstreamer-devel at lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
gstreamer-devel at lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


-- 
Thanks and Regards,Saurabh Bora

PH NO : 7038166900EMAIL : saurabh9bora at gmail.com             saurabh9bora at outlook.com

_______________________________________________
gstreamer-devel mailing list
gstreamer-devel at lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
gstreamer-devel at lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190314/56c39d0d/attachment-0001.html>