Regulate speed of decoded frames using presentation timestamp

[Nicolas Dufresne]

Le mardi 16 mars 2021 à 16:21 -0500, Andressio a écrit :
> Thank you for your reply
> 
> 
> Nicolas Dufresne-5 wrote
> > This return back in time indicates the presence of B-Frames. The decoder
> > will
> > reorder them for you. Notice the DTS, which should be monotonic before the
> > decoder.
> 
> I don't think there are B-frames in the stream. The return back in time
> happens a variable number of times: in my experiments 3 times per hour at
> most. I guess that if it was due to B-frames the jump back should be much
> more frequent and predictable. Consider that I'm using a keyframe interval
> of 10 at 25 fps.

Good point, then perhaps a clock skew in the jitterbuffer, best run with this
env set, so you get warnings from gst:

  GST_DEBUG=2

> 
> 
> Nicolas Dufresne-5 wrote
> > In GStreamer, decoders are not responsible for smoothing the transmission.
> > They
> > simply process as fast as possible. Dowstream the decoder, playback
> > pipelines
> > will usually contain a short raw video queue (which allow buffering when
> > decoding is faster).  After that queue, there is a display componenent,
> > (could
> > be glimagesink). These element are using GstBaseSink base class, which
> > implement
> > most of the synchronisation code. It will translate the PTS into
> > running-time,
> > and then translate this to clock time in order to determin how long to
> > wait
> > before displaying. Audio sink are much more complex.
> 
> I will look into GstBaseSink, thanks. Is relying on the PTS the best shot I
> have to regulate the output?

Well, running-time, which is obtain through:

running_time = gst_segment_to_running_time (segment, GST_FORMAT_TIME, pts);

The segment structure is obtain by watching for segment events, it is also
copied into GstSamples to make it easy for appsink users.

> 
> 
> Nicolas Dufresne-5 wrote
> > p.s. Remember that decoding complexity varies within a stream, so not all
> > frames
> > decode at equal speed. Some HW decoder would smooth this, but this is
> > atipical
> > for GPU decoders and PC software.
> 
> It is very interesting. Could you please provide some documentation about
> these different behaviours? 

I don't have much documentation to share, this is mostly accumulated knowledge.
In general, CODECs are composed of multiple compression tools, see each tools as
functions that need to be called with specific parameters and order. If a tool
isn't used, you have less functions to call, hence you get the output with less
clock cycles.

> 
> 
> 
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel