Issues when streaming SRT from RTSP sources.

[Daniel Pendse]

Hello,

I am encountering challenges while attempting to stream SRT from two RTSP sources. The setup involves handling two RTSP sources, one for video and the other for audio. These sources are managed by separate RTP bins. The video stream experiences roughly a delay of 2-3 seconds due to individual processing.

The RTSP streams are subjected to depay, parsing, and muxing using the mpegtsmux element, followed by streaming through srtsink. A series of queues and caps-filters are interspersed in this pipeline. The synchronisation is entrusted to Gstreamer, which appears to be functioning correctly.

However, I have seen instances where only the audio stream is received through SRT. I suspect this might occur when the video stream reaches the pipeline slightly after the audio stream as the pipeline already has been initialized? Therefore I wonder if anyone knows how to handle this sync issue for the mpegtsmux? Perhaps add blocking probes on pads?  buffer-waiting mechanisms? or are there more effective strategies?

Additionally, during testing, I’ve encountered occasional pipeline issues and/or SRT receiver errors. For instance, I’ve observed errors when attempting to play the stream using the playbin element (unsuccessfully!) 

```
GST_DEBUG=*:2,tsdemux:4,mpegtspacketizer:4 gst-launch-1.0 playbin uri=srt://:7878
-------------------------------------
0:00:00.507269000  5556 0x60000331aa30 WARN                  srtsrc gstsrtsrc.c:227:gst_srt_src_fill:<source> discont detected 12206790 (expected: 0)
0:00:00.530047000  5556 0x60000331aa30 INFO        mpegtspacketizer mpegtspacketizer.c:795:mpegts_try_discover_packet_size: have packetsize detected: 188 bytes
0:00:00.546111000  5556 0x60000330ebc0 WARN               h264parse gsth264parse.c:1525:gst_h264_parse_handle_frame:<h264parse0> broken/invalid nal Type: 1 Slice, Size: 5751 will be dropped
-----------------------------------
0:00:01.399142000  5556 0x60000331aa30 WARN                 tsdemux tsdemux.c:2751:gst_ts_demux_queue_data:<tsdemux0> warning: CONTINUITY: Mismatch packet 8, stream 2 (pid 0x0041)
WARNING: from element /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstTSDemux:tsdemux0: CONTINUITY: Mismatch packet 8, stream 2 (pid 0x0041)
Additional debug info:
../subprojects/gst-plugins-bad/gst/mpegtsdemux/tsdemux.c(2751): gst_ts_demux_queue_data (): /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstTSDemux:tsdemux0
—————————...
```

It’s noteworthy that the same stream is typically playable using FFplay.

Any ideas to what is causing the SRT playback issue? 

Furthermore, I’ve observed my pipeline occasionally entering a “stalled” state, with the RTPBin associated with the video source emitting on-sender-timeout signals. This seems to occur when attempting to stream SRT to a receiver that has stalled. Pretty sure I have seen this before where an RTPBin stops receiving data input. Can anyone explain more on why the RTPBin enters this state, emitting on-sender-timeout signals, and what they really mean?

I’ve struggled to replicate these issues using testsrc unfortunately.. 

Bunch of questions here.. I’m grateful for any help and guidance! 

Thank you,
Daniel Pendse