RTSP latency issues, effect of sync=false

[Nicolas Dufresne]

Le jeudi 23 juillet 2020 à 19:17 +0200, Guillermo Rodriguez Garcia a
écrit :
> Hello,
> 
> El mié., 22 jul. 2020 a las 22:03, Nicolas Dufresne
> (<nicolas at ndufresne.ca>) escribió:
> > Le mercredi 22 juillet 2020 à 13:21 +0200, Guillermo Rodriguez Garcia a
> > écrit :
> > > Hi all,
> > > 
> > > I am using GStreamer to fetch live video from IP cameras (RTSP /
> > > H.264). My application does not require audio.
> > > 
> > > In some cases I am seeing stuttering video with many dropped frames,
> > > and I see the typical "There may be a timestamping problem, or this
> > > computer is too slow" messages on GStreamer's debug output.
> > > 
> > > Nonetheless I don't think this is a performance issue in the receiver:
> > > Sometimes this happens for a low-bandwidth video stream (640x480, 15
> > > fps, < 1 Mbps), and the problems will go away if I actually _increase_
> > > the image resolution, frame rate, or bit rate in the camera itself.
> > 
> > That seems similar to the effect of burst in the network, like when
> > using wifi.
> 
> I see. Is there any way to verify this hypothesis?
> 
> Also.. if this (network issues) was the cause, why would the problem
> only happen for a given configuration, and then go away if I e.g.
> increase the image resolution (-> higher bandwidth requirements) ?

When you increase the resolution, you also increase the data rate. So
for the wifi case, which optimize throughput by bursting data, it means
the burst buffer will be more often full, so it will be sending data
more often, reducing the latency.

> 
> > > I have checked the following:
> > > - Adding queue elements between other elements in the pipeline doesn't
> > > seem to help
> > > - If I increase the latency parameter in the rtspsrc element (by
> > > default I am setting this to 200ms), this fixes the problem (but also
> > > introduces undesirable delays)
> > > - If I add sync=false to the sink, this also fixes the problem
> > > 
> > > The last option seems to always fix the problem without introducing
> > > any adverse side-effects (that I am aware of). My question is: Is
> > > there any reason not to use this? Anything I should bear in mind? Note
> > > that all streams are live, at any given time I am only using one
> > > source and one sink (not mixing anything), and I am not using audio at
> > > all.
> > 
> > With sync=false, the playback rate will be unstable during these
> > bursts. On some wifi, I could see clear freeze and speedup effect from
> > that. It will then get worst when things get congested.
> 
> I understand sync=false means "play incoming frames as soon as they
> arrive, ignoring the timestamp" (i.e. "don't discard frames that
> arrive too late"). Is this correct?

Correct.

> 
> I then assume that the speedup is caused by the sink rendering a bunch
> of frames that "arrived together"?

Together or closer by then what they were initially produced. Also
correct.

> 
> Is there any way to tell Gstreamer to "play incoming frames as soon as
> they arrive, i.e. don't discard frames that arrive too late, _but_ if
> there are multiple frames waiting, discard the older ones" ?

The sink element have no queues, so it does know if there is some late
frame already there. So I believe the answer is no.

> 
> > What happens is that the latency is a deadline, when the frames get
> > near that deadline, GStreamer try to play the "cachtup" game. But
> > catching up on live stream isn't something reliable or doable. In fact,
> > I've had better result with qos=0 for this case.
> 
> Just tried this (qos=0), but it doesn't seem to help.
> 
> > What advance apps can do is monitor CPU usage and buffer drops. If the
> > CPU usage is low, but you see a lot of buffer drops, you could
> > increasing the latency (this setting can be changed live). This would
> > effectively let you receiver adapt to the network conditions.
> 
> Understood, thank you.
> 
> > It's of course up to you, for some cases, smooth playback isn't
> > strictly required.
> 
> Thanks,
> 
> Guillermo Rodriguez Garcia
> guille.rodriguez at gmail.com
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel