RTSP latency issues, effect of sync=false

[Guillermo Rodriguez Garcia]

Hello,

El mié., 22 jul. 2020 a las 22:03, Nicolas Dufresne
(<nicolas at ndufresne.ca>) escribió:
>
> Le mercredi 22 juillet 2020 à 13:21 +0200, Guillermo Rodriguez Garcia a
> écrit :
> > Hi all,
> >
> > I am using GStreamer to fetch live video from IP cameras (RTSP /
> > H.264). My application does not require audio.
> >
> > In some cases I am seeing stuttering video with many dropped frames,
> > and I see the typical "There may be a timestamping problem, or this
> > computer is too slow" messages on GStreamer's debug output.
> >
> > Nonetheless I don't think this is a performance issue in the receiver:
> > Sometimes this happens for a low-bandwidth video stream (640x480, 15
> > fps, < 1 Mbps), and the problems will go away if I actually _increase_
> > the image resolution, frame rate, or bit rate in the camera itself.
>
> That seems similar to the effect of burst in the network, like when
> using wifi.

I see. Is there any way to verify this hypothesis?

Also.. if this (network issues) was the cause, why would the problem
only happen for a given configuration, and then go away if I e.g.
increase the image resolution (-> higher bandwidth requirements) ?

>
> >
> > I have checked the following:
> > - Adding queue elements between other elements in the pipeline doesn't
> > seem to help
> > - If I increase the latency parameter in the rtspsrc element (by
> > default I am setting this to 200ms), this fixes the problem (but also
> > introduces undesirable delays)
> > - If I add sync=false to the sink, this also fixes the problem
> >
> > The last option seems to always fix the problem without introducing
> > any adverse side-effects (that I am aware of). My question is: Is
> > there any reason not to use this? Anything I should bear in mind? Note
> > that all streams are live, at any given time I am only using one
> > source and one sink (not mixing anything), and I am not using audio at
> > all.
>
> With sync=false, the playback rate will be unstable during these
> bursts. On some wifi, I could see clear freeze and speedup effect from
> that. It will then get worst when things get congested.

I understand sync=false means "play incoming frames as soon as they
arrive, ignoring the timestamp" (i.e. "don't discard frames that
arrive too late"). Is this correct?

I then assume that the speedup is caused by the sink rendering a bunch
of frames that "arrived together"?

Is there any way to tell Gstreamer to "play incoming frames as soon as
they arrive, i.e. don't discard frames that arrive too late, _but_ if
there are multiple frames waiting, discard the older ones" ?

>
> What happens is that the latency is a deadline, when the frames get
> near that deadline, GStreamer try to play the "cachtup" game. But
> catching up on live stream isn't something reliable or doable. In fact,
> I've had better result with qos=0 for this case.

Just tried this (qos=0), but it doesn't seem to help.

>
> What advance apps can do is monitor CPU usage and buffer drops. If the
> CPU usage is low, but you see a lot of buffer drops, you could
> increasing the latency (this setting can be changed live). This would
> effectively let you receiver adapt to the network conditions.

Understood, thank you.

>
> It's of course up to you, for some cases, smooth playback isn't
> strictly required.

Thanks,

Guillermo Rodriguez Garcia
guille.rodriguez at gmail.com