[gst-devel] video playback fast (record with v4l2src)

[Thomas Vander Stichele]

Hi,


> A very enlightening post, at least to me.  The interesting conclusion I
> draw from this is that for video, the principle of "frame rate" is
> fundamentally broken.  It is just a restriction as to what timestamps
> are valid by allowing only equidistant values.  That must be the reason
> why advanced container formats like Matroska just store the timestamp of
> each frame, which is of course more flexible.  As far as I can see, this
> is also how video is handled inside GStreamer.  I now understand the
> "framerate" caps structure field of video caps as some sort of
> non-mandatory hint.

That is correct.  Real video with consumer devices needs to be
shoehorned using force into a notion of a stream with a constant
framerate.  The videorate element does a brute force approach by
dropping or doubling frames to respect a constant framerate, which is
then amenable to fitting it into a theora video codec (which is constant
framerate as well, sadly) or an avi container.

This loses a lot of valuable information.  To take a simple example,
assume you have two webcams, and you want to do picture in picture.
In a perfect world, you would be able to capture both streams, and save
their timestamps as well, no matter how erratic they are.  The two
streams are also unlikely to be capturing at the same interval, so none
of the frames of the first webcam would correlate with the frames of the
second webcam.

A playback application could then display this picture-in-picture at 50
Hz on your TV and do all the complex filtering/smoothing to get the best
possible result for that display.  But since you're typically forced to
mux into a format that has fixed framerate, you have to make these
complex tradeoffs *before* display, *and* somehow store the offset
between the two streams that end up in separate files.

Thomas