RTSP latency issues, effect of sync=false

Fri Jul 24 12:33:02 UTC 2020

El jue., 23 jul. 2020 a las 22:57, Nicolas Dufresne
(<nicolas at ndufresne.ca>) escribió:
>
> Le jeudi 23 juillet 2020 à 19:17 +0200, Guillermo Rodriguez Garcia a
> écrit :
> > Hello,
> >
> > El mié., 22 jul. 2020 a las 22:03, Nicolas Dufresne
> > (<nicolas at ndufresne.ca>) escribió:
> > > Le mercredi 22 juillet 2020 à 13:21 +0200, Guillermo Rodriguez Garcia a
> > > écrit :
> > > > Hi all,
> > > >
> > > > I am using GStreamer to fetch live video from IP cameras (RTSP /
> > > > H.264). My application does not require audio.
> > > >
> > > > In some cases I am seeing stuttering video with many dropped frames,
> > > > and I see the typical "There may be a timestamping problem, or this
> > > > computer is too slow" messages on GStreamer's debug output.
> > > >
> > > > Nonetheless I don't think this is a performance issue in the receiver:
> > > > Sometimes this happens for a low-bandwidth video stream (640x480, 15
> > > > fps, < 1 Mbps), and the problems will go away if I actually _increase_
> > > > the image resolution, frame rate, or bit rate in the camera itself.
> > >
> > > That seems similar to the effect of burst in the network, like when
> > > using wifi.
> >
> > I see. Is there any way to verify this hypothesis?
> >
> > Also.. if this (network issues) was the cause, why would the problem
> > only happen for a given configuration, and then go away if I e.g.
> > increase the image resolution (-> higher bandwidth requirements) ?
>
> When you increase the resolution, you also increase the data rate. So
> for the wifi case, which optimize throughput by bursting data, it means
> the burst buffer will be more often full, so it will be sending data
> more often, reducing the latency.

OK, makes sense (even though I am not using Wifi -- all devices are on
the same wired LAN)

> > > > I have checked the following:
> > > > - Adding queue elements between other elements in the pipeline doesn't
> > > > seem to help
> > > > - If I increase the latency parameter in the rtspsrc element (by
> > > > default I am setting this to 200ms), this fixes the problem (but also
> > > > introduces undesirable delays)
> > > > - If I add sync=false to the sink, this also fixes the problem
> > > >
> > > > The last option seems to always fix the problem without introducing
> > > > any adverse side-effects (that I am aware of). My question is: Is
> > > > there any reason not to use this? Anything I should bear in mind? Note
> > > > that all streams are live, at any given time I am only using one
> > > > source and one sink (not mixing anything), and I am not using audio at
> > > > all.
> > >
> > > With sync=false, the playback rate will be unstable during these
> > > bursts. On some wifi, I could see clear freeze and speedup effect from
> > > that. It will then get worst when things get congested.
> >
> > I understand sync=false means "play incoming frames as soon as they
> > arrive, ignoring the timestamp" (i.e. "don't discard frames that
> > arrive too late"). Is this correct?
>
> Correct.
>
> >
> > I then assume that the speedup is caused by the sink rendering a bunch
> > of frames that "arrived together"?
>
> Together or closer by then what they were initially produced. Also
> correct.
>
> >
> > Is there any way to tell Gstreamer to "play incoming frames as soon as
> > they arrive, i.e. don't discard frames that arrive too late, _but_ if
> > there are multiple frames waiting, discard the older ones" ?
>
> The sink element have no queues, so it does know if there is some late
> frame already there. So I believe the answer is no.

Uhm.. but then for the same reason, I would expect the "speedup
effect" to be rather limited.
I mean, since the sink has no queues (and in fact I am not using
queues anywhere else in the pipeline) there shouldn't be that too many
"late" frames to play -- only the bunch of frames that were sent "more
or less together" due to the network bursts you mentioned.

Anyway my problem seems to be that when the frames arrive, the sink
discards them because of the timestamp (i.e. this is not a performance
problem, but a latency problem). Using sync=false helps because the
sink will not discard frames, but is subject to the speedup problems
you describe.

Is there any way to configure the sink so that it does not drop late
frames (or at least to be a bit more lenient) without using sync=false
?

Thank you for the help and the feedback.

BR,

Guillermo

>
> >
> > > What happens is that the latency is a deadline, when the frames get
> > > near that deadline, GStreamer try to play the "cachtup" game. But
> > > catching up on live stream isn't something reliable or doable. In fact,
> > > I've had better result with qos=0 for this case.
> >
> > Just tried this (qos=0), but it doesn't seem to help.
> >
> > > What advance apps can do is monitor CPU usage and buffer drops. If the
> > > CPU usage is low, but you see a lot of buffer drops, you could
> > > increasing the latency (this setting can be changed live). This would
> > > effectively let you receiver adapt to the network conditions.
> >
> > Understood, thank you.
> >
> > > It's of course up to you, for some cases, smooth playback isn't
> > > strictly required.
> >
> > Thanks,
> >
> > Guillermo Rodriguez Garcia
> > guille.rodriguez at gmail.com
> > _______________________________________________
> > gstreamer-devel mailing list
> > gstreamer-devel at lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

-- 
Guillermo Rodriguez Garcia
guille.rodriguez at gmail.com