[Bug 766790] v4l2src generates wrong timestamps for encoded video.

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=766790

--- Comment #3 from Nicolas Dufresne (stormer) <nicolas at ndufresne.ca> ---
I just notice that your thread is about 1.4.5 on Beagle Bone. Have you tested
recent kernel and/or recent GStreamer ? I have taken out my C920, and I cannot
reproduce your result (GStreamer master and kernel 4.4). As expected, on older
kernel I expect that the bad-timestamp code triggers, and timestamp endup the
time when v4l2src manage to dequeue the frame. This might be a little jitterry,
but it's the best interpolation you can do. Adding a queue after v4l2src
usually remove the jitter. On newer kernel, the HW timestamp is ignored, hence
the timestamp is the time capture right after the IRQ is triggered (which is
accurate, minus the latency of course). This is live sources, so the moment the
frame is capture, is the timestamp. If your camera is jittery by design, you
should have jittery timestamp.

(In reply to Peter Maersk-Moller from comment #2)
> Can you point me to some file/functions where you do the bad time stamp
> detection and time stamp interpolation in v4l2src?

https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/sys/v4l2/gstv4l2src.c#n721

We don't do interpolation from existing timestamp. My opinion is that the
kernel should do that, because it gets much more accurate information. Though,
that code is broken in UVC driver and was disabled temporarily in 4.2 until
someone get the time to look at it. Obviously, if it turns out your firmware is
broken, complain to Logitech.

> 
> In my code, I read from a frame fifo at a constant frame rate and it works
> pretty well even out time stamps from all these buggy webcams. The encoded
> frames may have variable timed timestamps, but the frames them selves are
> captured at an steady rate (and subsequently should be palyed at a steady
> rate).

Not sure what your code is doing (share code if you want to keep that bug open
without providing solutions). I do have deployed code using the current v4l2src
(and also uvch264src) and it worked fine.

> 
> Anyway in uvch264src, you have the parameter 'fixed-framerate'. I would have
> expected that, when enabled, it would try to generate equal spaced time
> stamps.

This property will send UVC_H264_RATECONTROL_FIXED_FRM_FLG to the camera. If
your camera ignores it, complain to your vendor. I do remember Youness (this
element author) reporting that it only worked if you use H264 muxed into JPEG
(anything with a preview feed).

> 
> True that we should not break things. However, I also believe we should in
> general try to correct the timestamps, when possible. It could be a
> parameter you can enable so other things that don't need/want it can ignore
> it.

The only completely incorrect timestamp are those that go backward.
GStreamer/V4L2 does not let timestamp go backward for capture since 1.6, so if
this is the problem you see, update GStreamer, or backport the fixes.

> 
> True the firmware should be fixed in all these cameras, but that is not
> going to happen. True it should be fixed in the driver and or in V4L2
> community. Don't really know anyone there so that may not happen either.
> 
> True I can compensate for it in my software, although it would be really
> nice to have it done earlier.
> 
> One thought. h264parse does, as I understand try to correct some TS errors.
> It could happen there as well. Something like 'videorate', but for a H264
> stream although adding frames is most likely not possible and throwing away
> frames can cause problems. Admittedly I haven't thought it through. But best
> would be to be able to correct timestamps in v4l2src or alterntively in the
> uvch264src using v4l2src.

You need at least one valid timestamp (DTS or PTS) to fix the timestamp in
h264parse. At the moment, you seem to state that we have none, so this won't
work. Again, smoothing timestamp can be wrong, as it may not be representative
to the capture time. There is load of people that do cares about capture time
accuracy.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.