RTSP latency issues, effect of sync=false

[Nicolas Dufresne]

Le mercredi 22 juillet 2020 à 13:21 +0200, Guillermo Rodriguez Garcia a
écrit :
> Hi all,
> 
> I am using GStreamer to fetch live video from IP cameras (RTSP /
> H.264). My application does not require audio.
> 
> In some cases I am seeing stuttering video with many dropped frames,
> and I see the typical "There may be a timestamping problem, or this
> computer is too slow" messages on GStreamer's debug output.
> 
> Nonetheless I don't think this is a performance issue in the receiver:
> Sometimes this happens for a low-bandwidth video stream (640x480, 15
> fps, < 1 Mbps), and the problems will go away if I actually _increase_
> the image resolution, frame rate, or bit rate in the camera itself.

That seems similar to the effect of burst in the network, like when
using wifi.

> 
> I have checked the following:
> - Adding queue elements between other elements in the pipeline doesn't
> seem to help
> - If I increase the latency parameter in the rtspsrc element (by
> default I am setting this to 200ms), this fixes the problem (but also
> introduces undesirable delays)
> - If I add sync=false to the sink, this also fixes the problem
> 
> The last option seems to always fix the problem without introducing
> any adverse side-effects (that I am aware of). My question is: Is
> there any reason not to use this? Anything I should bear in mind? Note
> that all streams are live, at any given time I am only using one
> source and one sink (not mixing anything), and I am not using audio at
> all.

With sync=false, the playback rate will be unstable during these
bursts. On some wifi, I could see clear freeze and speedup effect from
that. It will then get worst when things get congested.

What happens is that the latency is a deadline, when the frames get
near that deadline, GStreamer try to play the "cachtup" game. But
catching up on live stream isn't something reliable or doable. In fact,
I've had better result with qos=0 for this case.

What advance apps can do is monitor CPU usage and buffer drops. If the
CPU usage is low, but you see a lot of buffer drops, you could
increasing the latency (this setting can be changed live). This would
effectively let you receiver adapt to the network conditions.

It's of course up to you, for some cases, smooth playback isn't
strictly required.

> 
> Thank you,
> 
> Guillermo Rodriguez Garcia
> guille.rodriguez at gmail.com
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel