[pulseaudio-discuss] VLC, PulseAudio and large tlengths

[Rémi Denis-Courmont]

Le samedi 20 août 2011 21:46:16 Tanu Kaskinen, vous avez écrit :
> > Anyway. Ignoring underflows is not really an option. When they do
> > _really_ happen, e.g. due to serious scheduling problems, VLC has to
> > resynchronize the stream somehow. I don't see any solution other than
> > down-sampling or padding, or is there?
> 
> By resynchronizing I guess you mean maintaining A-V sync?

Mostly lip sync, yes.

> (...) I think the idea is that you can at any time query the current
> playback latency (fixed hardware latency + currently buffered data)
> and use this information to schedule the video frames.

That would arguably be the best way to implement a video file player.
But the display vertical refresh is an alternative master clock. In the first 
case, you may need to drop or duplicate frames. In the second case, you may 
need to resample the audio signal.

Anyway VLC is built with live playback in mind (it started as a DVB-IP 
receiver afterall). VLC uses to the input signal as the master clock (or the 
CPU monotonic clock by default). I believe gstreamer uses a similar logic 
though I have not checked. In fact, that is the only practical option if the 
receiver does not control the input pace.

So the audio can and does drift. This is compensated through resampling. 
Normally VLC would do it internally. Now the PulseAudio is unique among VLC 
audio outputs insofar as PulseAudio resamples on VLC behalf. David suggested 
that a while ago.

> I'm not sure how downsampling is relevant here. Is the video being
> synchronized to the wall clock instead of the audio clock and you need
> to make the audio stream go faster to catch up with the video stream?

Currently, VLC tolerates 40 ms advance and 60 ms delay as per EBU 
Recommendation 37. If a PulseAudio latency update indicates that playback does 
not fall within that 100 ms sliding window, VLC changes the sample rate to try 
to restore synchronization without glitch.

It is thus essential that the stream gets triggered approximately on time, 
whether that is initially, upon resuming from pause, or upon recovering from 
underflow. Otherwise, resampling kicks in and you get to hear Doppler.

> If
> that's the case, don't you run into trouble with the wall clock and the
> audio clock drifting apart (the sound card most likely doesn't run
> exactly at 48000 Hz even if it claims to do so)?

Oh yeah. We do. But that's unavoidable in the general case.

-- 
Rémi Denis-Courmont
http://www.remlab.net/
http://fi.linkedin.com/in/remidenis