GStreamer clocks and latency calculation

[Bob Barker]

Thanks Tim. You're right: we are taking the documentation  literally
because we're new to GSteamer and need to write our own elements.  Without
a tutorial, there's nothing else to on.


A few more questions:

Is the decoding timestamp the time the buffer actually was decoded (after
the decoding has finished) or the time it should be decoded (as in "this
needs to be decoded at <some point in the future>)


Same question as above  about presentation timestamp.



On latency the documentation says:*

"14.6.1. Latency compensation
*

Before the pipeline goes to the PLAYING state, it will, in addition to
selecting a clock and calculating a

base-time, calculate the latency in the pipeline. It does this by doing a
LATENCY query on all the sinks

in the pipeline. The pipeline then selects the maximum latency in the
pipeline and configures this with a

LATENCY event.

All sink elements will delay playback by the value in the LATENCY event.
Since all sinks delay with the

same amount of time, they will be relative in sync."



When does each element calculate its latency and when does the  pipeline
issue the latency  query?  Doign it in the playing state seems too late.
Does it happen in the preroll state? Some buffers have to be processed by
every element to determine where the longest latency is.


On Sat, Oct 5, 2013 at 2:42 AM, Tim-Philipp Müller <t.i.m at zen.co.uk> wrote:

> On Fri, 2013-10-04 at 12:41 -0700, Bob Barker wrote:
>
> Hi,
>
> > We’re new to GStreamer and are writing our own elements.  We’re trying
> > to understand which clocks are used for jitter calculation. From the
> > GStreamer documentation:
> >
> >
> http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-qos.txt
> >
> > “A buffer with timestamp B1 arrives in the sink at time T1. The buffer
> > timestamp is then synchronized against the clock which yields a jitter
> > J1 return value from the clock. (snip)"
> >
> >
> > We assume the clock is the monotonically increasing wall clock, not
> > the running time nor the stream time shown ins section  14.4 of the
> > GStreamer application development manual. From the documentation the
> > running time always starts at zero so the running time and stream time
> > are always less than the clock time.
> >
> > In the GStreamer clocks diagram (and the real-world) the clock time is
> > always greater than stream time and the running time. Given that and
> > the equation above (  J1 = CT – B1) , jitter will always be positive.
> > According to GStreamer documentation “If the jitter is positive
> > however, the entry arrived too late in the sink and should therefore
> > be dropped. “
> >
> > Questions:
> >
> > 1.      So all buffers will always be late and should be dropped?!
> >  What are we missing?
>
> It sounds like maybe you're taking these docs a little too literal. The
> clock time is not necessarily the actual value gst_clock_get_time()
> returns, and the buffer time stamp will likely be adjusted for things
> like the configured latency, and converted into running time for
> synchronization purposes.
>
> Things also depend a bit on the exact pipeline, and what elements are
> involved. If you have filesrc ! parse ! dec ! audiosink and the pipeline
> selects the audio clock from the sink, then that clock will be based on
> audio samples rendered (and some of your assumptions about the real
> world are not necessarily true any more).
>
>
> > 2.      Which timestamp is the documentation referring to?
> > GStreamer calls out one in the Linux version of  gstbuffer.h:“
> > GstClockTime           timestamp;” and  two in the windows version:
> > pts and gts.
>
> It depends on the circumstances. Usually you will sync on pts, but there
> may be cases where one would sync on dts if it is available, e.g. for
> encoded content.
>
> There is no difference between "the Linux version" and "the Windows
> version". What you're seeing is the old 0.10 version vs. the new 1.0
> version, which have different APIs.
>
>
> > 3.      Which of the three clocks (clocktime, running time, stream
> > time) are the pts and dts referring to?
>
> Those are not "three clocks", there is only one clock. pts/dts are
> "stream time", their actual values need to be used in connection with
> the SEGMENT to transform those into running time for synchronization
> purposes.
>
>
> > 4.      It seems the correct jitter formula is:
> >  Jitter  = buffer_arrival_time - base_time – buffer_time_stamp
> > Where buffer_arrival_time is read from the system clock.
>
> I believe the clock time as mentioned in the document is supposed to be
> already adjusted like that (i.e. time pipeline spent in playing state
> according to the selected clock) and not an actual clock time value.
>
> Cheers
>  -Tim
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20131008/342cfce2/attachment.html>