[gst-devel] Dropped frames at gstbaseaudiosrc

[Mark Nauwelaerts]

Sebastian Voecking wrote:
> Hello!
> 
> Thanks for your quick reply.
> 
>> What to do about it (possibly);
>> allow for "looser/more" threading and/or buffering, e.g.
>> - use queue in one or both of the streams going into the muxer
>> (e.g. before the encoder element)
> 
>> - simlarly, alsasrc (and osssrc) properties buffer-time and latency-time
>> determine the total buffer size and individual buffer size
> 
> I tried that and now I am here:
> 
> gst-launch-0.10 v4l2src device="/dev/video0" !
> video/x-raw-yuv,width=640,height=480 ! queue ! theoraenc ! oggmux
> name="ogg" alsasrc device="hw:1,0" buffer-time=40000000
> latency-time=400000 ! audio/x-raw-int,rate=32000 ! queue !
> audioconvert ! vorbisenc ! ogg. ogg. ! filesink location=test.ogg
> 
> No more frame get dropped now. But the resulting video and audio are
> both very choppy.
> 
> What else should I try? BTW I am using Ubuntu Feisty Fawn here with
> gstreamer 0.10.12.

A bit hard to reproduce as hardware may play quite a role in this.
The following works fairly well in my case:

v4l2src device="/dev/video0" ! \
video/x-raw-yuv,width=384,height=284 ! ffenc_mpeg4 ! avimux \
name="ogg" alsasrc  buffer-time=40000000 \
latency-time=400000 ! audio/x-raw-int,rate=32000 ! \
audioconvert ! lame ! ogg. ogg. ! filesink location=test.avi

(ok, some misnaming in avimux name=ogg)

Why the changes:
* it seems my (not that old) hardware can't keep up with real-time theora
encoding (or I am doing something wrong in the theora area)
* queue has been dropped because the mmap'ed buffers that v4l2src output
(somehow??) choke my (threading) performance further down (strange but real)

Quite likely, not all will be fully OK:
* I get quite some "ffenc_mpeg4: failed to encode buffer" reports.
This is due to threading/buffering; fairly regularly 2 video buffers will happen
to be processed in rapid succession, leading to identical timestamps, and the
encoder does not like this
* in the 0.10.5 version, only 2 (IIRC) video frame buffers are allocated; this
will probably lead to (some) frame-dropping, and either choppy video or
video-audio out-of-sync

What to do about this;
* newer v4l2src (queue-size) allows allocating more frame buffers; should
prevent dropping at that level.  In turn, however, this will lead to even more
"clustering" in the timestamps of the output buffers (and encoder complaints)
* this "clustering" can be somewhat counter-acted by using the videorate
element.  Although that should then result in pretty good audio-video sync
(without complaints), video may be a bit choppier, since videorate will respond
to the "clustering" by (lot of) dropping frames one moment, and (lot of)
duplicating frames other moments.

(again; also see the link given earlier, which has some other talk along these
lines; I do eventually manage to get really good capturing, but it takes some
attention/caution)

Regards,
Mark