[pulseaudio-discuss] a question about audio synchronization between local sink and tunnel sink

[Sun, Xiaodong]

Hi, Tanu,

Yes, with my embedded sink device. I can get synchronized playback eventually. But the time between playing start and synchronization is too long. The minimum is 1.5 minutes (sorry 1.5s is a typo error). So it is not good enough for synchronized playback.

Now there are two directions to improve this. The first (more feasible but not perfect) is tweaking the adjust_rates() function and make synchronization faster when latency is bigger. The second (don't know how much effort but should be a perfect solution) is adding a new algorithm based on timestamp besides current rate adjust algorithm. 

I will try the first one since it is a feasible method for me to try.  For the second one, how much effort do you think there is? And do you have any plan to do this kind of improvement?

Thanks!


Xiaodong

-----Original Message-----
From: Tanu Kaskinen [mailto:tanuk at iki.fi] 
Sent: Wednesday, October 17, 2012 6:08 AM
To: Sun, Xiaodong
Cc: pulseaudio-discuss at lists.freedesktop.org
Subject: Re: [pulseaudio-discuss] a question about audio synchronization between local sink and tunnel sink

On Mon, 2012-10-15 at 17:42 +0000, Sun, Xiaodong wrote:
> Hi, Tanu,
> 
> There is about 0.5s constant delay between tunnel sink and local sink 
> and these two sinks are never in-sync no matter how long I play. I 
> also tried different values for module-combine-sink adjust_time 
> parameter and it doesn't help.
> 
> Then, I tried another setup by replacing tunnel sink PC-2 using 
> another embedded audio device. The symptom is different. The tunnel 
> sink (embedded audio device) lags local sink (PC-1) at the beginning.
> Then after playing a while, these two sinks can be in-sync eventually.
> The period from un-sync to sync differs when different adjust_time is 
> applied. The shortest time is 1.5s, when adjust_time is set to 1. I 
> traced the logs, and found that during this period, either the latency 
> of one of the sinks is too much different from the other or adjusted 
> rate of one of the sinks jumps back and forth around the target sample
> rate.   
> 
> I looked at the source code of module-combine-sink.c. The
> adjust_rates() function seems to try to reduce sample rate of the sink 
> with smaller latency through resampling module, in order to increase 
> that sink's latency and eventually make the latency of both sinks 
> same.

This will eventually make the streams synchronized, so it's a good approach.

> This algorithm chooses to stop doing rate adjustment when the 
> difference between sinks are too much different (more than 
> ~0.2*adjust_time seconds).

I guess you're referring to this code:

        if (new_rate < (uint32_t) (base_rate*0.8) || new_rate > (uint32_t) (base_rate*1.25)) {
            pa_log_warn("[%s] sample rates too different, not adjusting (%u vs. %u).", o->sink_input->sink->name, base_rate, new_rate);
            new_rate = base_rate;
        }

It may be that whoever wrote that didn't take large initial latency differences into account, so this sanity check might need some tweaking.

> And also the new adjusted rate can not exceed 0.2% of the current 
> rate.

This ensures that it's not audible when the rate changes. This shouldn't break anything, though - it will only make synchronizing time longer (and possibly lead to underruns, but that's not necessarily a bad thing, since if the latency difference is large, it's maybe a good idea to drop some audio to reach synchronization faster).

> All these logics seem telling me that this rate adjustment algorithm 
> is designed to provide synchronizations between sinks with only minor 
> latency difference, which are mainly caused by Audio DAC sampling 
> clock deviation on different sink devices, not for devices with major 
> latency difference, which could be caused by network or buffering in 
> audio S/W stack.

Yes, I think it's mainly designed for handling clock deviation issues.
But it shouldn't be unreasonably difficult to change the code to handle also large initial latency differences.

> For synchronized playback with major latency difference, a common 
> timestamp (e.g. PTS in AV sync) should be used. This timestamp is sent 
> to all sinks before real playback starts. All sinks can only buffer 
> audio sample data, but can not start playback until the local time 
> matches timestamp. The local timer synchronization can be achieved by 
> either NTP or PTP protocol. Does pulseaudio include such or similar 
> scheme for combined sink devices' synchronization playback?

No, there's no support for that. Synchronized playback start requires another algorithm (something like what you suggested) in addition to the algorithm that ensures that the streams stay synchronized even with deviating clocks.

--
Tanu