Gracefully Handle (or ignore) GstFlowReturns

Wed Jul 8 16:34:11 UTC 2020

*TL;DR - How do I intercept and handle a bad GstFlowReturn (GST_FLOW_ERROR)
to prevent stopping the pipeline, without modifying the plugin source code?
I am using C++ on MSYS2 using GStreamer 1.16.2.*

I'm trying to process an MJPEG stream from an IP camera starting with the
following GStreamer pipeline:

/souphttpsrc location="http://192.168.18.101/mjpeg" is-live=true retries=-1
! queue leaky=2 max-size-buffers=10 ! multipartdemux boundary=fbdr !
multiqueue ! jpegparse ! videorate ! 'image/jpeg, framerate=20/1' ! jpegdec
! d3dvideosink sync=false async=false/

Whether using gst-launch or the c++ program I'm writing, I am able to get a
smooth live view of the camera, and this works until multipartdemux receives
a bad header. When this occurs, it returns a GST_FLOW_ERROR which travels to
the bus event listener and eventually kills the pipeline. The pipeline may
run flawlessly for minutes or hours before this occurs, making debugging
difficult.

How can I intercept and handle GstFlowReturns? I expect bad frames every now
and then since the cameras stream images over a network connection, so
errors are expected and shouldn't be fatal. Ideally, I would drop the bad
buffer and allow videorate to replace the missing frame. The closest thing I
could find is here: 
http://gstreamer-devel.966125.n4.nabble.com/Ignoring-errors-of-fdsink-td972387.html
<http://gstreamer-devel.966125.n4.nabble.com/Ignoring-errors-of-fdsink-td972387.html> 
. Most internet searches return results for finding the cause of a
GST_FLOW_ERROR, not handling one.

I tried adding a probe to the multipartdemux sink pad to listen for events
(GST_PAD_PROBE_TYPE_EVENT_BOTH ), but the errors aren't caught by this. I've
also tried removing the multipartdemux and multiqueue from the pipeline, but
that just pushes the error downstream to jpegdemux.

Environment is Windows 10. Pipeline is launched either using gst-launch-1.0
or C++ code compiled in MSYS2. GStreamer version is 1.16.2 (x64) installed
using MSYS2's pacman. Application log is attached, output from gst-launch
(--gst-debug=3) is below. 

*Background for the C++ program log*
The program creates 4 identical pipelines to connect to 4 cameras. There are
extra tees and branches built in to allow linking of processing/recording
branches as needed, but these were not enabled when the error occured. GLib
is used in general and for logging, and d3dvideosink windows are captured
into a GTK+3 GUI. This particular log shows it running for almost 16 hours
without error. 2 of the 4 cameras failed when I came in, but this was
probably by chance, not due to interaction (I was on my laptop on a separate
network, no one else was in the office). I doubt this is a bandwidth problem
since the error occurs when streaming 1 camera (gst-launch, around 50Mbps)
or 4 cameras (around 200 Mbps) with no appreciable change in error
frequency.

*Logs*
$ gst-launch-1.0.exe --gst-debug=3 souphttpsrc
location="http://192.168.18.101/mjpeg?res=half" is-live=true retries=-1 !
queue leaky=2 max-size-buffers=2 ! multipartdemux boundary=fbdr ! multiqueue
! jpegparse ! videorate ! 'image/jpeg, framerate=20/1' ! jpegdec ! queue !
d3dvideosink

0:00:00.020476900  2124        34454b0 WARN      GST_PLUGIN_LOADING
gstplugin.c:792:_priv_gst_plugin_load_file_for_registry: module_open failed:
'C:\msys64\mingw64\lib\gstreamer-1.0\libgstdvdread.dll': The specified
module could not be found.

(gst-launch-1.0:2124): GStreamer-WARNING **: 09:25:59.204: Failed to load
plugin 'C:\msys64\mingw64\lib\gstreamer-1.0\libgstdvdread.dll':
'C:\msys64\mingw64\lib\gstreamer-1.0\libgstdvdread.dll': The specified
module could not be found.
0:00:00.023569700  2124        34454b0 WARN      GST_PLUGIN_LOADING
gstplugin.c:792:_priv_gst_plugin_load_file_for_registry: module_open failed:
'C:\msys64\mingw64\lib\gstreamer-1.0\libgstresindvd.dll': The specified
module could not be found.

(gst-launch-1.0:2124): GStreamer-WARNING **: 09:25:59.207: Failed to load
plugin 'C:\msys64\mingw64\lib\gstreamer-1.0\libgstresindvd.dll':
'C:\msys64\mingw64\lib\gstreamer-1.0\libgstresindvd.dll': The specified
module could not be found.
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
0:00:00.108345400  2124        34454b0 WARN               structure
gststructure.c:1861:priv_gst_structure_append_to_gstring: No value transform
to serialize field 'session' of type 'SoupSession'
Got context from element 'souphttpsrc0': gst.soup.session=context,
session=(SoupSession)NULL, force=(boolean)false;
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:02.440240200  2124        2521a80 FIXME           videodecoder
gstvideodecoder.c:944:gst_video_decoder_drain_out:<jpegdec0> Sub-class
should implement drain()
0:00:04.402283600  2124        2521900 WARN                basesink
gstbasesink.c:3003:gst_base_sink_is_too_late:<d3dvideosink0> warning: A lot
of buffers are being dropped.
0:00:04.402307600  2124        2521900 WARN                basesink
gstbasesink.c:3003:gst_base_sink_is_too_late:<d3dvideosink0> warning: There
may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstD3DVideoSink:d3dvideosink0:
A lot of buffers are being dropped.
Additional debug info:
../gstreamer-1.16.2/libs/gst/base/gstbasesink.c(3003):
gst_base_sink_is_too_late ():
/GstPipeline:pipeline0/GstD3DVideoSink:d3dvideosink0:
There may be a timestamping problem, or this computer is too slow.
0:00:06.173914500  2124        2521960 WARN          multipartdemux
multipartdemux.c:505:multipart_parse_header:<multipartdemux0> error:
Boundary not found in the multipart header
0:00:06.174099900  2124        25219c0 WARN                 basesrc
gstbasesrc.c:3072:gst_base_src_loop:<souphttpsrc0> error: Internal data
stream error.
0:00:06.174137000  2124        25219c0 WARN                 basesrc
gstbasesrc.c:3072:gst_base_src_loop:<souphttpsrc0> error: streaming stopped,
reason error (-5)
0:00:06.174165400  2124        25219c0 WARN                   queue
gstqueue.c:988:gst_queue_handle_sink_event:<queue0> error: Internal data
stream error.
0:00:06.174192700  2124        25219c0 WARN                   queue
gstqueue.c:988:gst_queue_handle_sink_event:<queue0> error: streaming
stopped, reason error (-5)
ERROR: from element
/GstPipeline:pipeline0/GstMultipartDemux:multipartdemux0: Could not
demultiplex stream.
Additional debug info:
../gst-plugins-good-1.16.2/gst/multipart/multipartdemux.c(505):
multipart_parse_header ():
/GstPipeline:pipeline0/GstMultipartDemux:multipartdemux0:
Boundary not found in the multipart header
Execution ended after 0:00:06.069975900
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

CapProc_2020-07-07_16-46-54.log
<http://gstreamer-devel.966125.n4.nabble.com/file/t379531/CapProc_2020-07-07_16-46-54.log>  

--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/