Issues trying to synchronise video appsrc with audio

[Sebastian Dröge]

On So, 2014-05-18 at 23:06 +0200, stic at free.fr wrote:
> Hi,
> 
> I am trying to synchronise video produced by appsrc with audio.
> 
> Here is the pipeline I am using:
> appsrc name=vsource ! video/x-h264,framerate=30/1 ! h264parse config-interval=2 ! queue ! muxer. openslessrc name=asource ! audioconvert ! audio/x-raw,rate=16000 ! faac ! queue ! matroskamux name=muxer streamable=true ! tcpserversink...
> 
> The problem is that as soon as I introduce audio in the pipeline it blocks video and shows following warnings:
> gstaudiobasesrc.c:863:gst_audio_base_src_create:<asource> warning: Can't record audio fast enough
> gstaudiobasesrc.c:863:gst_audio_base_src_create:<asource> warning: Dropped 10560 samples. This is most likely because downstream can't keep up and is consuming samples too slowly.
> 
> Here is how I produce timestamps in the function called by the "need-data" signal:
> // timestamp is initialized with 0 value
> static GstClockTime timestamp = 0;
> 
> //then for each buffer the timestamp value is updated like this:
> GST_BUFFER_PTS(buffer) = timestamp;
> GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, framerate);
> timestamp += GST_BUFFER_DURATION(buffer);
> 
> I know this is probably not the best way to give timestamps to buffers, but it works very well with video alone.
> I tried to replace this by using do-timestamp=true and is-live=true properties to appsrc but as soon as I try to do this it ends with following warning from matroskamux for every buffer: "Invalid buffer timestamp; dropping buffer"

That way of giving timestamps to the buffers is not really wrong and
should work fine the first time you use the plugin (the second time it
won't obviously as the static variable will have the value from the
previous run). Assuming that you actually produce frames in that
framerate, and also note that you accumulate rounding errors in the
timestamps there.

As you say that audio-only works just fine, I would guess that your
problem with the "can't keep up fast enough" is that the queue
downstream of the decoder runs full, and then the live audio source gets
stuck... because you don't produce the video fast enough, which will
block the muxer and have the muxer consume data slower than the audio
source produces.

-- 
Sebastian Dröge, Centricular Ltd - http://www.centricular.com
Expertise, Straight from the Source
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 966 bytes
Desc: This is a digitally signed message part
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20140519/e6528748/attachment.sig>