time of day timestamps handling in gstreamer (v4l2 source)

[Philippe De Muyter]

On Wed, Aug 27, 2014 at 10:09:46AM -0400, Nicolas Dufresne wrote:
>
>
> Le mer 27 aoû 2014 à 9:46, Philippe De Muyter <phdm at macq.eu> a écrit :
>> One can clearly see that the pts field contains only some offset from the
>> pipeline start time, computed as 'abs_time - base_time - delay' in
>> gst_v4l2src_create, and not really related to the timestamp given in
>> the v4l2_buffer.
>> I did not find any option that I could give to either gst-launch or 
>> v4l2src
>> to change that behaviour.
>> Also, I do not undestand your statement : "this only works if the 
>> timestamps
>> are generated from the POSIX monotonic clock or the system clock".  Here 
>> the
>> timestamps given by the v4l2 driver are clearly generated from the system
>> clock.
>
> For new drivers, it's mandatory to use MONOTONIC time, only unmaintained 
> drivers (or old kernel) will give you time-of-the-day. We do our best to 
> detect this, but we should implement support for the new MONOTONIC flag in 
> order to ensure we keep these as intact as possible:
>
> https://bugzilla.gnome.org/show_bug.cgi?id=732911
>
> The timestamp is indeed offset from the pipeline base_time, otherwise they 
> would not correlate with the segment. If I was to revisit this, I think I 
> would prefer pushing the correct segment start, but this mean we need to 
> delay some action after we have the first frame. For the case the rate is 
> in fact dynamic (most cases), we would have to delay even further in order 
> to get the correct frame duration. There has been discussion already but we 
> haven't concluded yet.
>
> About the delay, I'm not convince that the delay calculation we do is right 
> as the frame might have remained queued in the driver while we are pushing 
> downstream. The timestamp would not longer represent the end of capture 
> time (note that flags where introduced to tell what the timestamp stands 
> for, this could also be added). I doubt we have a strict definition for 
> capture timestamp in GStreamer at the moment though.
>
> An additional request has been made recently to expose the V4L2 sequence 
> number. Some driver expose a sequence number which aims at doing frame 
> perfect synchronisation (special HW that speak to each other), this could 
> also improve things a bit if we find a way to integrate that.

How can a source having known timeofday timestamps (perhaps even old timestamps
as with dv1394 or matroska files) ensure that they are preserved and made
available for the sink ?

Philippe