<div dir="auto"><div> <div class="gmail_quote"><div dir="ltr">Le sam. 21 avr. 2018 12:12, Erickson.LU <<a href="mailto:ericksonlushan@gmail.com">ericksonlushan@gmail.com</a>> a écrit : </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I tried to use appsink get frame and convert it cv::mat and use imshow show the mat on screen. The program some part is from opensource code. I have 2 question : (1)I think gst_buffer_ref(buffer); used for releasing the Buffer. But why i can't use gst_buffer_unref(buffer); ??? (2) The program use appsink pull frame Buffer from v4l2src and convert it to cv::mat, at last use imshow show the mat. But when i comment out *g_main_loop_thread_ = boost::thread(RunMainLoop); * Everything is ok . The mat will continuously show the mat, and the log "frame count = ..." will scroll . The log is : *main thread lwpid = 20051 main thread tid = 1807194688 Process_frame thread lwpid = 20055 Process_frame thread tid = 1377380096 frame count = 1 Element APP_SINK0 changed state from NULL to READY. Element capsfilter1 changed state from NULL to READY. Element videoconvert0 changed state from NULL to READY. Element capsfilter0 changed state from NULL to READY. Element v4l2src0 changed state from NULL to READY. Element pipeline0 changed state from NULL to READY. Element capsfilter1 changed state from READY to PAUSED. Element videoconvert0 changed state from READY to PAUSED. Element capsfilter0 changed state from READY to PAUSED. STREAM STATUS received from element v4l2src0: CREATE Element v4l2src0 changed state from READY to PAUSED. Element pipeline0 changed state from READY to PAUSED. Unparsed message received of type: new-clock Element capsfilter1 changed state from PAUSED to PLAYING. Element videoconvert0 changed state from PAUSED to PLAYING. Element capsfilter0 changed state from PAUSED to PLAYING. Element v4l2src0 changed state from PAUSED to PLAYING. STREAM STATUS received from element v4l2src0: ENTER Unparsed message received of type: stream-start Element APP_SINK0 changed state from READY to PAUSED. Unparsed message received of type: async-done Element APP_SINK0 changed state from PAUSED to PLAYING. Element pipeline0 changed state from PAUSED to PLAYING. frame count = 2 frame count = 3 frame count = 4 frame count = 5 frame count = 6 frame count = 7 frame count = 8 frame count = 9 frame count = 10 ...* But when I don't comment out *g_main_loop_thread_ = boost::thread(RunMainLoop); * The mat just show only one frame on my screen and pause at there. And the log is: *main thread lwpid = 18025 main thread tid = 4097198656 RunningMainLoop thread lwpid = 18027 RunningMainLoop thread tid = 3675776768 Element APP_SINK0 changed state from NULL to READY. Element capsfilter1 changed state from NULL to READY. Element videoconvert0 changed state from NULL to READY. Element capsfilter0 changed state from NULL to READY. Element v4l2src0 changed state from NULL to READY. Element pipeline0 changed state from NULL to READY. Element capsfilter1 changed state from READY to PAUSED. Element videoconvert0 changed state from READY to PAUSED. Element capsfilter0 changed state from READY to PAUSED. STREAM STATUS received from element v4l2src0: CREATE Element v4l2src0 changed state from READY to PAUSED. Element pipeline0 changed state from READY to PAUSED. STREAM STATUS received from element v4l2src0: ENTER Unparsed message received of type: new-clock Process_frame thread lwpid = 18029 Element capsfilter1 changed state from PAUSED to PLAYING. Process_frame thread tid = 3658991360 Unparsed message received of type: stream-start Element videoconvert0 changed state from PAUSED to PLAYING. Element capsfilter0 changed state from PAUSED to PLAYING. Element v4l2src0 changed state from PAUSED to PLAYING. Element APP_SINK0 changed state from READY to PAUSED.Unparsed message received of type: async-done Element APP_SINK0 changed state from PAUSED to PLAYING. frame count = 1 Element pipeline0 changed state from PAUSED to PLAYING. test_now: Fatal IO error 11 (Resource temporarily unavailable) on X server :0.* If i use *RunMainLoop(); instead of g_main_loop_thread_ = boost::thread(RunMainLoop);* on the main thread . It will block at *g_main_loop_run(loop);*. I can understand at blocking at there. But why i run it at a new thread it will block the Process_frame method at cv::waitKey(30); ? Does it the function *g_main_loop_run* will occupy the X Windows server or something, make opencv can't use that ? The whole program is bleow: GstElement* pipeline; GstBus* bus; boost::thread g_main_loop_thread_; GstElement* appsink; boost::mutex mutex; GstSample* retrievedBuffer; GstSample* currentBuffer; #define SINK0 "v4l2src device=/dev/video0 ! video/x-raw,format=(string)I420, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1 ! videoconvert ! video/x-raw,format=(string)BGR, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1 ! appsink name=APP_SINK0 caps=video/x-raw,format=BGR,width=640,height=480,pixel-aspect-ratio=1/1" const cv::Scalar RED(0,0,255), GREEN(0,255,0); void DoNewSample() { GstSample* sample = gst_app_sink_pull_sample(GST_APP_SINK(appsink)); if (sample) { boost::lock_guard<boost::mutex> guard(mutex); if (currentBuffer != 0) { gst_sample_unref(currentBuffer); } currentBuffer = sample; } } bool IsNewFrameAvailable() { boost::lock_guard<boost::mutex> guard(mutex); return ((retrievedBuffer == 0) && (currentBuffer != 0)); } bool GetSample(GstSample** sample) { boost::lock_guard<boost::mutex> guard(mutex); if (retrievedBuffer == 0) { if (currentBuffer != 0) { retrievedBuffer = currentBuffer; currentBuffer = 0; (*sample) = retrievedBuffer; retrievedBuffer=0; } } else { (*sample) = NULL; return false; } return true; } void ReleaseSample() { boost::lock_guard<boost::mutex> guard(mutex); if (retrievedBuffer) gst_sample_unref(retrievedBuffer); retrievedBuffer = 0; } void Process_frame(){ GstSample* sample; GstMapInfo map; GstStructure* s; GstBuffer* buffer; GstCaps* caps; GstMemory* mem; int height, width, size; int count=0; void* map_data; g_print("Process_frame thread lwpid = %u\n", syscall(SYS_gettid)); g_print("Process_frame thread tid = %u\n", pthread_self()); while(true){ if(IsNewFrameAvailable()){ GetSample(&sample); caps = gst_sample_get_caps(sample); buffer = gst_sample_get_buffer(sample); s = gst_caps_get_structure(caps, 0); gst_structure_get_int(s, "height", &height); gst_structure_get_int(s, "width", &width); size = gst_buffer_get_size(buffer); if (gst_buffer_map (buffer, &map, GST_MAP_READ)) { cv::Mat frame(height, width, CV_8UC3,map.data, cv::Mat::AUTO_STEP); std::string msg ="Copy Right"; cv::Point origin(0,0+13); cv::putText( frame, msg, origin, 1, 1, RED); count++; g_print("frame count = %d\n",count); cv::imshow("View_CV",frame); cv::waitKey(30); gst_buffer_unmap (buffer, &map); gst_buffer_ref(buffer); } gst_sample_unref (sample); ReleaseSample(); } else { } } } void EndOfStreamCallback(GstAppSink* appsink, gpointer user_data) { } GstFlowReturn PrerollCallback(GstAppSink* appsink, gpointer user_data) { GstSample* sample = gst_app_sink_pull_preroll(appsink); gst_sample_unref(sample); return GST_FLOW_OK; } GstFlowReturn SampleCallback(GstAppSink* appsink, gpointer user_data) { DoNewSample(); return GST_FLOW_OK; } bool GstMessageParser(GstBus* bus, GstMessage* msg, pipelinewrapper* pipeline) { if (msg != NULL) { GError* err = 0; gchar* debug_info = 0; switch (GST_MESSAGE_TYPE(msg)) { case GST_MESSAGE_ERROR: gst_message_parse_error(msg, &err, &debug_info); g_printerr("Error received from element %s: %s\n", GST_OBJECT_NAME(msg->src), err->message); g_printerr("Debugging information: %s\n", debug_info ? debug_info : "none"); g_clear_error(&err); g_free(debug_info); break; case GST_MESSAGE_WARNING: gst_message_parse_warning(msg, &err, &debug_info); g_printerr("Warning received from element %s: %s\n", GST_OBJECT_NAME(msg->src), err->message); g_printerr("Debugging information: %s\n", debug_info ? debug_info : "none"); g_clear_error(&err); g_free(debug_info); break; case GST_MESSAGE_INFO: gst_message_parse_info(msg, &err, &debug_info); g_printerr("Info received from element %s: %s\n", GST_OBJECT_NAME(msg->src), err->message); g_printerr("Debugging information: %s\n", debug_info ? debug_info : "none"); g_clear_error(&err); g_free(debug_info); break; case GST_MESSAGE_EOS: g_print("End-Of-Stream reached.\n"); break; case GST_MESSAGE_STATE_CHANGED: GstState old_state, new_state; gst_message_parse_state_changed(msg, &old_state, &new_state, 0); g_print("Element %s changed state from %s to %s.\n", GST_OBJECT_NAME(msg->src), gst_element_state_get_name(old_state), gst_element_state_get_name(new_state)); break; case GST_MESSAGE_QOS: break; case GST_MESSAGE_STREAM_STATUS: GstStreamStatusType stream_status_type; GstElement* owner; const gchar* stream_status_type_string; gst_message_parse_stream_status(msg, &stream_status_type, &owner); switch (stream_status_type) { case GST_STREAM_STATUS_TYPE_CREATE: stream_status_type_string = "CREATE"; break; case GST_STREAM_STATUS_TYPE_ENTER: stream_status_type_string = "ENTER"; break; case GST_STREAM_STATUS_TYPE_LEAVE: stream_status_type_string = "LEAVE"; break; case GST_STREAM_STATUS_TYPE_DESTROY: stream_status_type_string = "DESTROY"; break; case GST_STREAM_STATUS_TYPE_START: stream_status_type_string = "START"; break; case GST_STREAM_STATUS_TYPE_PAUSE: stream_status_type_string = "PAUSE"; break; case GST_STREAM_STATUS_TYPE_STOP: stream_status_type_string = "STOP"; break; } g_printerr("STREAM STATUS received from element %s: %s\n", GST_OBJECT_NAME(owner), stream_status_type_string); //g_free (stream_status_type_string); break; default: g_printerr("Unparsed message received of type: %s\n", gst_message_type_get_name(GST_MESSAGE_TYPE(msg))); break; } } return true; } void RunMainLoop() { GMainLoop* loop = g_main_loop_new(NULL, FALSE); </blockquote></div></div><div dir="auto"> </div><div dir="auto">This is wrong, Null mean default main context, which must be created on the same thread. It's really hard to control in which thread the one will be created. In your case, it probably get created in your main function, when you do gst_bus_add_watch(). You could try to create it here, but in general, it's more robust to create a new one. Nd then make everything explicit. Using push_thread_default, and also gst_bus_create_watch and g_source_attach</div><div dir="auto"> </div><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> g_print("RunningMainLoop thread lwpid = %u\n", syscall(SYS_gettid)); g_print("RunningMainLoop thread tid = %u\n", pthread_self()); g_main_loop_run(loop); } int main (int argc, char *argv[]) { gst_init(&argc, &argv); // XInitThreads(); boost::thread g_main_loop_thread0; GstAppSinkCallbacks appsinkCallbacks; std::string pipelineString = SINK0; std::string appsink_name="APP_SINK0"; GError* err = 0; pipeline = gst_parse_launch(pipelineString.c_str(), &err); bus = gst_pipeline_get_bus(GST_PIPELINE(pipeline)); gst_bus_add_watch(bus, (GstBusFunc)GstMessageParser, NULL); g_main_loop_thread_ = boost::thread(RunMainLoop); // RunMainLoop(); appsink = gst_bin_get_by_name(GST_BIN(pipeline), appsink_name.c_str()); g_print("main thread lwpid = %u\n", syscall(SYS_gettid)); g_print("main thread tid = %u\n", pthread_self()); appsinkCallbacks.new_preroll = &PrerollCallback; appsinkCallbacks.new_sample = &SampleCallback; // appsinkCallbacks.eos = &EndOfStreamCallback; gst_app_sink_set_drop(GST_APP_SINK(appsink), true); gst_app_sink_set_max_buffers(GST_APP_SINK(appsink), 1); gst_app_sink_set_callbacks(GST_APP_SINK(appsink), &appsinkCallbacks, NULL, NULL); GstStateChangeReturn ret = gst_element_set_state(pipeline, GST_STATE_PLAYING); if (ret == GST_STATE_CHANGE_FAILURE) { g_printerr("Unable to set the pipeline to the playing state"); gst_object_unref(pipeline); return false; } // Process_frame(); g_main_loop_thread0 = boost::thread(&Process_frame); g_main_loop_thread0.join(); /* cleanup and exit */ gst_element_set_state(pipeline,GST_STATE_NULL); gst_object_unref (pipeline); exit (0); } -- Sent from: <a href="http://gstreamer-devel.966125.n4.nabble.com/" rel="noreferrer noreferrer" target="_blank">http://gstreamer-devel.966125.n4.nabble.com/</a> _______________________________________________ gstreamer-devel mailing list <a href="mailto:gstreamer-devel@lists.freedesktop.org" target="_blank" rel="noreferrer">gstreamer-devel@lists.freedesktop.org</a> <a href="https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel" rel="noreferrer noreferrer" target="_blank">https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel</a> </blockquote></div></div></div>