Mixing v4l2 sources with compositor

[Nicolas Dufresne]

Le jeudi 21 mai 2015 à 11:43 +0200, Frédéric Sureau a écrit :
> I have noticed (using identity) that v4l2src buffers do not start pts 
> at 
> 0:00:00.000000000.
> First source starts pts at approximately 140ms which correspond to 7 
> frames at 50 fps.
> This is exactly the number of black frames at the beginning.
> In v4l2src, pts is calculated from buffer absolute timestamp (coming 
> from v4l2 driver) and v4l2src PLAYING state absolute time. The 
> difference between these 2 absolute times gives pts.

The V4L2 sources take some time to initialize, meanwhile the clock is
running. If you bring back the timestamp 0 zero, and both sources
didn't take the same amount of time to initialize, the end result won't
be synchronized. Arguably, videorate should not break synchronization,
but that's likely another subject.

I think a nice way to solve your case, would be to add a first frame
strategy to compositor (something configurable). Right now (in live) it
should be producing frames on a deadline without any special cases. I
think it could be nice to have a mode that will wait for all sinkpads
to have received a frame before producing the first output. That may
require dropping frames. An intermediate mode, that would wait for at
least 1 frame could also be nice.

Nicolas