Issues trying to synchronise video appsrc with audio

[stic at free.fr]

please can you give me advices on how I can synchronise correctly video and audio in my case ?
Do you know if I have to insert empty buffers to compensate the fact that the appsrc video input does not push data when the input is not changing ?

Thanks

----- Mail original -----
De: stic at free.fr
À: "Discussion of the development of and with GStreamer" <gstreamer-devel at lists.freedesktop.org>
Envoyé: Lundi 19 Mai 2014 18:48:06
Objet: Re: Issues trying to synchronise video appsrc with audio

yes you're probably right.
If I set the audio queue with queue max-size-time=0 then the video is not blocked but everything is very delayed, and audio is more and more delayed with the time.
Increasing buffer-time on the audio source also seems to result to the same kind of behaviour.

Your answer makes me also think that the video appsrc does not produce buffers when the input does not change, so this is probably why what you say is right and it does not produce enough video buffers.
But even when I try to insert empty buffers it is not better ... 

is that the right way to do ?
If so what should be set as timestamp and duration for empty buffers ?

Thank you.


----- Mail original -----
De: "Sebastian Dröge" <sebastian at centricular.com>
À: "Discussion of the development of and with GStreamer" <gstreamer-devel at lists.freedesktop.org>
Envoyé: Lundi 19 Mai 2014 09:24:00
Objet: Re: Issues trying to synchronise video appsrc with audio

On So, 2014-05-18 at 23:06 +0200, stic at free.fr wrote:
> Hi,
> 
> I am trying to synchronise video produced by appsrc with audio.
> 
> Here is the pipeline I am using:
> appsrc name=vsource ! video/x-h264,framerate=30/1 ! h264parse config-interval=2 ! queue ! muxer. openslessrc name=asource ! audioconvert ! audio/x-raw,rate=16000 ! faac ! queue ! matroskamux name=muxer streamable=true ! tcpserversink...
> 
> The problem is that as soon as I introduce audio in the pipeline it blocks video and shows following warnings:
> gstaudiobasesrc.c:863:gst_audio_base_src_create:<asource> warning: Can't record audio fast enough
> gstaudiobasesrc.c:863:gst_audio_base_src_create:<asource> warning: Dropped 10560 samples. This is most likely because downstream can't keep up and is consuming samples too slowly.
> 
> Here is how I produce timestamps in the function called by the "need-data" signal:
> // timestamp is initialized with 0 value
> static GstClockTime timestamp = 0;
> 
> //then for each buffer the timestamp value is updated like this:
> GST_BUFFER_PTS(buffer) = timestamp;
> GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, framerate);
> timestamp += GST_BUFFER_DURATION(buffer);
> 
> I know this is probably not the best way to give timestamps to buffers, but it works very well with video alone.
> I tried to replace this by using do-timestamp=true and is-live=true properties to appsrc but as soon as I try to do this it ends with following warning from matroskamux for every buffer: "Invalid buffer timestamp; dropping buffer"

That way of giving timestamps to the buffers is not really wrong and
should work fine the first time you use the plugin (the second time it
won't obviously as the static variable will have the value from the
previous run). Assuming that you actually produce frames in that
framerate, and also note that you accumulate rounding errors in the
timestamps there.

As you say that audio-only works just fine, I would guess that your
problem with the "can't keep up fast enough" is that the queue
downstream of the decoder runs full, and then the live audio source gets
stuck... because you don't produce the video fast enough, which will
block the muxer and have the muxer consume data slower than the audio
source produces.

-- 
Sebastian Dröge, Centricular Ltd - http://www.centricular.com
Expertise, Straight from the Source

_______________________________________________
gstreamer-devel mailing list
gstreamer-devel at lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
gstreamer-devel at lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel