Video synchronization

[Tim-Philipp Müller]

On Mon, 2012-05-14 at 01:35 -0700, enricom wrote:

Hi Enrico,

> I would like to know if there's a way to extract the buffer timestamp
> of the video stream that a video player is playing at a certain moment. So
> far I've been using the timestamps extracted by an identity element placed
> just before the video player but the handoff signals are not generated
> synchronously, so between 2 buffers of length = 0.04 s there's a system time
> delta extracted at the handoff signal generation slightly different than
> 0.04 s...for this reason I suppose that either the video player receives
> these buffers asynchronously, so it has a buffer itself and synchronizes
> with the general clock, or the identity signal is emitted asynchronously (or
> maybe both)...
> For audio/video synchronization purposes I need to know something like: in a
> certain precise instant the video player is displaying a frame with
> timestamp=x..is it possible?
> Thank you very much.

What exactly is it that you would like to do / achieve?

I presume just querying the current playback position using:

  GstFormatTime format = GST_FORMAT_TIME;
  gin64 duration;
  if (gst_element_query_position (pipeline, &format, &dur)) {
    g_print ("Position: %" GST_TIME_FORMAT "\n", GST_TIME_ARGS (dur));
  } else {
    g_print ("Could not query position.\n");
  }

(once the pipeline is prerolled, i.e. in paused/playing state, or you
have received an ASYNC_DONE message on the bus) is not what you're
after?

You can also get the current buffer from the video-sink using the
"last-buffer" property and check GST_BUFFER_TIMESTAMP() on it, but the
timestamp alone is not particularly useful without other information
like the newsegment event.

The handoff callback from the identity element is called synchronously
from the streaming thread btw, but once it goes to the videosink it is
usually not played right away, but the sink will wait for the right time
to render it, so the handoff signal will usually get the buffer too
early (unless the decoders or other upstream parts are too slow).

Cheers
 -Tim