[gst-devel] video playback fast (record with v4l2src)

[René Stadler]

Am Dienstag, den 19.09.2006, 21:24 +0200 schrieb Ronald S. Bultje:
[...]
> Look, v4l/v4l2 are not like a file, where (average) framerate can be
> guessed or approximated in advance. It's live. For TV cards, a good
> approximation is the norm framerate, which is 25 for PAL or 30/1.001 for
> NTSC, for example (if you don't drop frames while capturing). Now, for
> webcams, none of this is true anymore. However, the only (poor) API for
> doing this is a hack in some window flag field in the v4l1 API. v4l2 has
> no decent way of doing this at all. Since avimux uses the approximated
> framerate in advance, and we know that this value is totally screwed up
> for v4l1/2 for webcams, we know in advance that this will give problems.
> 
> There is no perfect way to do this, especially not if you want to capture
> audio as well. There are several things that could be done to improve the
> situation:
> * capture two frames for webcams and use the delta-T as a framerate
> approximation.
> * you could use videorate to insert/drop frames and thereby generate a
> stream of constant framerate as requested in the caps.
> * if you don't use audio, avimux could use n-frames / total_dur as a
> framerate (in the re-written header) at the end of capture. Of course,
> when capturing audio also, this will screw up sync and is thus not
> recommended as a general solution.
> * you could fix drivers and implement a v4l2 api to tell us the framerate
> that the device will approximate. Drivers generally know this better than
> us, since they are in direct contact with the USB bus, and can reserve a
> certain amount of datarate for themselves (and thus assert a certain
> framerate, to a certain extent).
> * avimux could be fixed to allow insert/drop frames (empty packets in the
> index) to approximate a framerate if the input isn't exactly the specified
> framerate.
> 
> There's probably other things that I didn't think of that could be done,
> also.
> 
> Good luck,
> Ronald
> 

A very enlightening post, at least to me.  The interesting conclusion I
draw from this is that for video, the principle of "frame rate" is
fundamentally broken.  It is just a restriction as to what timestamps
are valid by allowing only equidistant values.  That must be the reason
why advanced container formats like Matroska just store the timestamp of
each frame, which is of course more flexible.  As far as I can see, this
is also how video is handled inside GStreamer.  I now understand the
"framerate" caps structure field of video caps as some sort of
non-mandatory hint.

Looks like AVI is not appropriate for capturing from video live sources
at all.

Regards,
  René