[gst-devel] video playback fast (record with v4l2src)

[Ronald S. Bultje]

Hi Jesse,

On Sep 20, 2006, at 4:22 AM, Jesse Lang wrote:
> How should I capture both audio and video so that they sync?  I  
> imagine it needs to be muxed in one form or another.   If AVI is  
> not an option as you say, and OGG does not work either, what  
> options are there?

I don't think anyone ever tests this. Note that there are no  
applications (apart from Fluendo's Flumotion) that use any of those  
features, +/-90% of gst-based apps are simple audio players. Two  
things are important for this kind of work: Firstly, the inputs  
should support syncing to a non-self clock. I think v4l1/2src support  
this (I wrote them like that), although it may not be default  
behaviour anymore. The audio source should then either provide the  
pipeline clock (which is what I used for testing), or should sync to  
another clock, also (not supported). Then, secondly, the output  
format that you use should support frame skips (this is mostly  
important for video), or you should use videorate and audiorate to  
workaround this for the formats that don't (thus generating frame  
inserts/drops).

So, your options are, theoretically:
- use videorate + audiorate, check options on vl1/2src to ensure that  
it syncs to pipeline clock. This will not give the best result (it  
will contain frame drops/inserts), especially for webcams, but should  
be reasonable (because it's not too big a problem) for tv cards.
- do not use videorate + audiorate, use a format that supports  
timestamps and does not depend on framerate (e.g. ogg, mpeg, asf, I  
think quicktime can do this too?). Make sure that this is implemented  
and properly functioning in the gstreamer elements that you're using.  
If not (as is probably the case for you), fix that. For you, this  
probably comes down to a bit of debugging in the source elements to  
ensure proper timestamp outputting and proper clock handling, and  
then fix the output muxers (oggmux for ogg, ffmpegmux for mpeg/ 
quicktime, asfmux for asf; you can probably also add asf to  
ffmpegmux) to handle timestamps rather than the (wrong) fps in the caps.

HTH,
Ronald