time of day timestamps handling in gstreamer (v4l2 source)

[Philippe De Muyter]

Hello Sebastian,

On Tue, Aug 26, 2014 at 09:33:25AM +0300, Sebastian Dröge wrote:
> On Mo, 2014-08-25 at 18:06 +0200, Philippe De Muyter wrote:
> > Hi,
> >=20
> > I have an existing application using v4l2 cameras on different computers
> > synchronized by ntp.  The output (frames) of the different computers
> > are timestamped with the ntp time of the snapshot, provided by the v4l2
> > driver.
> >=20
> > I can guarantee that the ntp time has no hickups (at least not when it ma=
> tters).
> >=20
> > I would like to rewrite this application using gstreamer.  How can I
> > get the same quality timestamps at the output of my gstreamer pipeline ?
> > In other words : in which buffer attribute(s) can I retrieve that timesta=
> mp ?
> 
> The buffer pts will contain the timestamps as provided via the timestamp
> field of struct v4l2_buffer. However this only works if the timestamps
> are generated from the POSIX monotonic clock or the system clock
> (gettimeofday()) currently, see the code in
> gstv4l2src.c:gst_v4l2src_create().

This is not what I see.  The kernel driver gives timeofday timestamps,
(verified by adding a printf after the VIDIOC_DQBUF ioctl returns)
but the gstreamer buffer has its pts field set to the offset from a
base time (pipeline start time ?)

this is the pipe-line I use for testing :

  gst-launch-1.0 -v v4l2src ! fakesink silent=false

And here is the output I get (with additional prints added by me after any
time-related assignment in gst_v4l2_buffer_pool_dqbuf and gst_v4l2src_create) :

  Setting pipeline to PAUSED ...
  Pipeline is live and does not need PREROLL ...
  /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event   ******* (fakesink0:sink) E (type: stream-start (10254), GstEventStreamStart, stream-id=(string)2b7f5679e9f0cee0776e7c9d96ffc3b4682b26f7b8931798b4254fc99bdbde9b, flags=(GstStreamFlags)GST_STREAM_FLAG_NONE, group-id=(uint)0;) 0x119c968
  Setting pipeline to PLAYING ...
  /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = "video/x-bayer\,\ width\=\(int\)1280\,\ height\=\(int\)1024\,\ framerate\=\(fraction\)60/1"
  /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event   ******* (fakesink0:sink) E (type: caps (12814), GstEventCaps, caps=(GstCaps)"video/x-bayer\,\ width\=\(int\)1280\,\ height\=\(int\)1024\,\ framerate\=\(fraction\)60/1";) 0x119c9b0
  /GstPipeline:pipeline0/GstFakeSink:fakesink0.GstPad:sink: caps = "video/x-bayer\,\ width\=\(int\)1280\,\ height\=\(int\)1024\,\ framerate\=\(fraction\)60/1"
  New clock: GstSystemClock
  gst_v4l2_buffer_pool_dqbuf : timestamp = 1409139198905042000
  gst_v4l2_buffer_pool_dqbuf : output.timestamp = 1409139198905042000
  gst_v4l2src_create: timestamp = 1409139198905042000
  gst_v4l2src_create: gstnow = 87940655623010
  gst_v4l2src_create: gstnow-bis = 1409139198905389000
  gst_v4l2src_create: delay = gstnow - timestamp = 347000
  gst_v4l2src_create: abs_time = 87940655617677
  gst_v4l2src_create: base_time = 87940558180998
  gst_v4l2src_create: timestamp = abs_time - base_time = 97436679
  gst_v4l2src_create: timestamp > delay => timestamp -= delay
  gst_v4l2src_create: v4l2src->ctrl_time = 16666666
  gst_v4l2src_create: GST_BUFFER_TIMESTAMP (*buf) = 97089679
  /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event   ******* (fakesink0:sink) E (type: segment (17934), GstEventSegment, segment=(GstSegment)"GstSegment, flags=(GstSegmentFlags)GST_SEGMENT_FLAG_NONE, rate=(double)1, applied-rate=(double)1, format=(GstFormat)GST_FORMAT_TIME, base=(guint64)0, offset=(guint64)0, start=(guint64)0, stop=(guint64)18446744073709551615, time=(guint64)0, position=(guint64)0, duration=(guint64)18446744073709551615;";) 0x119c9f8
  /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = preroll   *******
  /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain   ******* (fakesink0:sink) (1310720 bytes, dts: 0:00:00.000000000, pts: 0:00:00.097089679, duration: 0:00:00.016666666, offset: 0, offset_end: 1, flags: 00002040 discont delta-unit ) 0x75d0c840

One can clearly see that the pts field contains only some offset from the
pipeline start time, computed as 'abs_time - base_time - delay' in
gst_v4l2src_create, and not really related to the timestamp given in
the v4l2_buffer.

I did not find any option that I could give to either gst-launch or v4l2src
to change that behaviour.

Also, I do not undestand your statement : "this only works if the timestamps
are generated from the POSIX monotonic clock or the system clock".  Here the
timestamps given by the v4l2 driver are clearly generated from the system
clock.

Thanks for any hint

Philippe

-- 
Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 Bruxelles