[gst-devel] video playback fast (record with v4l2src)

[Ronald S. Bultje]

Hi Edgar,

On Wed, 20 Sep 2006, Edgard Lima wrote:
> The problem is because the timestamps aren't too accurate  (would it be
> incremented exactly  1001/30000)?

Indeed, many drivers simply use a kernel implementation of gettimeofday()
for the timestamp, which isn't very accurate. I'd say this is
driver-specific though, if you do this inside the interrupt handler, it
shouldn't be too big a deal (I can't recall having big problems in my
driver, but I may be wrong here).

> Buffers coming from v4l2 drivers has timestamps and sequence numbers. If
> drivers implements sequence numbers, could it be used somehow?

Yes. Note that sequence numbers, at least in some drivers, do _not_ always
take lost frames into account! This means that you need a max-latency
setting (or something similar) inside v4l2src to decide when a frame is to
be considered lost and when you should manually update the sequence
number.

> The audio comes from alsa input, is there some way v4l2 drivers could
> also put timestamps to audio coming from tuner to make A/V sync?

They could, indirectly, by implementing a clock in v4l2src and then have
alsasrc use this clock for choosing which timestamp to put on buffers. In
my experience, the other way around gives better results, though, and this
is what the current behaviour is (i.e. alsasrc provides clock, v4l2src
syncs to this clock and generates timestamps based on the clock's time).

> If the answer to my first question is yes. I think I could write a
> smarter v4l2src that would increment the value of each time time-stamp
> exactly fps_d/fps_n (1001/30000 for this example).
> Or, it could be also done in videorate (videorate could wait a bit more
> before insert a new buffer) some thing like 'max-lateness' for 'alsasink'.
>
> What do you think?

Both sound useful. The v4l2src with smart timestamps is gonna be tricky,
since different drivers use the sequence number in a different way. Keep
that in mind when writing such a patch.

As for the videorate, that'll be easier to implement, so I'd give that a
go first.

However, in the end, please note that if alsasrc uses its own samplecount
for timestamp generation and v4l2src uses the v4l2_buffer timestamp field,
then they do _not_ use the same clock, and will thus slowly drift apart.
To prevent this, make sure that v4l2src uses the v4l2_buffer values _only_
to decide when a frame was lost, and uses the clock's time value for
timestamp generation on the buffer it outputs.

Ronald