GStreamer clocks and latency calculation

[Tim-Philipp Müller]

On Fri, 2013-10-04 at 12:41 -0700, Bob Barker wrote:

Hi,

> We’re new to GStreamer and are writing our own elements.  We’re trying
> to understand which clocks are used for jitter calculation. From the
> GStreamer documentation:
> 
> http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-qos.txt
> 
> “A buffer with timestamp B1 arrives in the sink at time T1. The buffer
> timestamp is then synchronized against the clock which yields a jitter
> J1 return value from the clock. (snip)"
>  
> 
> We assume the clock is the monotonically increasing wall clock, not
> the running time nor the stream time shown ins section  14.4 of the
> GStreamer application development manual. From the documentation the
> running time always starts at zero so the running time and stream time
> are always less than the clock time.
>
> In the GStreamer clocks diagram (and the real-world) the clock time is
> always greater than stream time and the running time. Given that and
> the equation above (  J1 = CT – B1) , jitter will always be positive.
> According to GStreamer documentation “If the jitter is positive
> however, the entry arrived too late in the sink and should therefore
> be dropped. “ 
> 
> Questions:
> 
> 1.      So all buffers will always be late and should be dropped?!
>  What are we missing?

It sounds like maybe you're taking these docs a little too literal. The
clock time is not necessarily the actual value gst_clock_get_time()
returns, and the buffer time stamp will likely be adjusted for things
like the configured latency, and converted into running time for
synchronization purposes.

Things also depend a bit on the exact pipeline, and what elements are
involved. If you have filesrc ! parse ! dec ! audiosink and the pipeline
selects the audio clock from the sink, then that clock will be based on
audio samples rendered (and some of your assumptions about the real
world are not necessarily true any more).


> 2.      Which timestamp is the documentation referring to?
> GStreamer calls out one in the Linux version of  gstbuffer.h:“
> GstClockTime           timestamp;” and  two in the windows version:
> pts and gts. 

It depends on the circumstances. Usually you will sync on pts, but there
may be cases where one would sync on dts if it is available, e.g. for
encoded content.

There is no difference between "the Linux version" and "the Windows
version". What you're seeing is the old 0.10 version vs. the new 1.0
version, which have different APIs.


> 3.      Which of the three clocks (clocktime, running time, stream
> time) are the pts and dts referring to?

Those are not "three clocks", there is only one clock. pts/dts are
"stream time", their actual values need to be used in connection with
the SEGMENT to transform those into running time for synchronization
purposes.


> 4.      It seems the correct jitter formula is:
>  Jitter  = buffer_arrival_time - base_time – buffer_time_stamp     
> Where buffer_arrival_time is read from the system clock.

I believe the clock time as mentioned in the document is supposed to be
already adjusted like that (i.e. time pipeline spent in playing state
according to the selected clock) and not an actual clock time value.

Cheers
 -Tim