[pulseaudio-discuss] Gap, silence and latency

[James Lee]

Tanu Kaskinen <tanuk <at> iki.fi> writes:

> 
> On Fri, 2012-03-16 at 17:34 +0000, James Lee wrote:
> > A/V sync is handled in the decoders based on the current PCR and PTS values 
for 
> > each audio/video frame.  Our concern is audio latency through pulse, not A/V 
> > synchronization.  The latency jumps right after the gap.
> 
> I'd still say that your problem is A/V sync, not latency, unless the
> latency information from Pulseaudio is incorrect. Doing A/V sync
> requires you to know how long it will take for the audio to come out of
> the speakers and for the video frames to be visible if you write data
> now, and make timing decisions based on that. If you do your timing
> decisions only based on the PCR and PTS values, it sounds like you
> expect to hear the audio immediately when you issue a write command in
> your code.
> 

Again, take pulseaudio out and everything is fine (with just ALSA).  Also, 
streaming an audio file to two clients shows that one with pulse lags behind 
after the gap.

> That said, it sounds like the assumption works well enough for you when
> you play the first movie. I don't really see a reason why continuing
> after a stream underrun should cause any longer delays than starting a
> new stream, unless there's prebuffering involved in the underrun case.
> If there's prebuffering in the underrun case, I'd expect there to be
> similar prebuffering also in the stream creation case, though...
> 

There's always prebuffering.  The same movie is being played 24/7 repeatedly.  
You turn (join the multicast) to that channel and start watching.  I am guessing 
the "first" one is always OK because the data is already being streamed the 
moment you turn to the channel.  Turning to the channel right when the gap 
starts should exhibit the same behavior.

> I guess you have checked that the A/V sync is actually still ok when the
> previous movie ends, and it only goes wrong when the next one starts? If
> you don't resample to compensate the drift between the server clock and
> the sound card clock, data can accumulate slowly in the stream buffer.
> Then again, stopping writing data during the gap between the movies
> should drain any accumulated data and reset the situation...
> 

There is always latency because of buffering.  The problem is that the latency 
grows right after the gap (period during which no data is coming in).

> I guess this was not very helpful in finding a solution. It sounds like
> there may be a bug in Pulseaudio or the alsa plugin. You can file a bug
> if you wish. A test program that shows that the latency is misbehaving
> would be great.
> 

As I stated above, this is a pulseaudio issue.

> Assuming that Pulseaudio reports the latency correctly, it should be
> also possible for you to fix your application to monitor the latency and
> adapt to jumps as needed. Also recreating the stream after the gap could
> be a simple workaround?
> 

Unfortunately, we do not know how long the gap is going to be.  At this point, 
we are leaning towards doing resampling and mixing ourselves.