Sound stopping (pipeline sync issue?)

[Tvrtko Ursulin]

Hi Marco,

Thank you for replying.

On Sunday 05 Jun 2011 07:45:24 Marco Ballesio wrote:
> On Thu, Jun 2, 2011 at 2:45 PM, Tvrtko Ursulin
> <tvrtko.ursulin at onelan.co.uk> wrote:
> > Hi all,
> > 
> > I have a test pipeline where sound works very unreliable unless sync
> > property on alsasink is set to false and I am wondering about why, what
> > is happening under the hood and so on.
> > 
> > Pipeline looks like this:
> > 
> > gst-launch-0.10 v4l2src device=/dev/video0 ! video/x-raw-
> > yuv,width=1920,height=1080,framerate=50/1 ! queue ! deinterlace ! queue !
> > xvimagesink alsasrc device=hw:2,0 provide-clock=false ! audio/x-raw-
> > int,rate=48000 ! queue ! alsasink
> 
> here the video processing is performed at quite high frame rate and
> resolution. Even though no actual processing is performed on the
> frames, memory copies are performed here and there, so I'd check the
> CPU load to begin with.

This seems to be fine, gst-launch load is 10-15% of a single core.
 
> > What is happening is that sound stops less than a second after pipeline
> > is launched and stays off. Video works fine.
> > 
> > If I add "sync=false" to alsasink then it all works fine. "provide-clock"
> > on the alsasrc has no effect there, I added it assuming video clock must
> > be a better choice?
> 
> In your case, as the pipeline is live and no sources would be
> providing a clock, the pipeline would use the system one:
> 
> http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/Gst
> Pipeline.html
> 
> (v4l2src doesn't provide a clock).
> 
> It's usually a bad choice, as it could lead to (short) audio dropouts
> when compensating skews. As audio is (almost) a continuous signal, a
> drop there is more annoying than in the case of video.

Right, so you suggest leaving provide-clock to true for alsasrc on the 
rationale that audio clock is more reliable than the system clock? Or some 
other reason?

> > My main question would be is "sync=false" on the audio output sync in any
> > way dangerous or not recommended?
> 
> it prevents the audio sink from dropping frames when they arrive too
> late, either because of audio skew or for delayed scheduling in the
> audio pipeline (or the cumulative effect of both). The intermediate
> queue risks to delay audio further because of more context switches (a
> queue "splits" the pipe in two threads), but here opinions may differ.

sync=false prevents dropping frame or sync=true?

As I said with sync=true I get no sound at all after the initial sub second 
period. This is even when alsasrc and alsasink are on the same device. 
 
> You might try and play with the alsa buffer size by increasing both
> "buffer-time" and "latency-time". Please note that this would increase
> the overall latency in your pipeline and, being it a live stream, it
> could not be what you really want.

Neither increasing or decreasing these by order of magnitude has any 
observable effect.

> > How does GStreamer synchronise video and audio with this kind of pipeline
> > and how does "sync=false" on audio sink affect that?
> 
> "grep"ping (or "find"ing) on the gstreamer core sources would have
> given you a quicker answer ;) . btw here you go:
> 
> http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-synch
> ronisation.txt

Well it is a bit complicated but in essence it says that things should 
synchronize, right? :)

Even with "disjoint" streams in a pipeline (not two streams coming for a 
demuxer which would then output correct timestamps for each)?

Many thanks,

Tvrtko