[pulseaudio-discuss] a question about audio synchronization between local sink and tunnel sink

[Tanu Kaskinen]

On Mon, 2012-10-15 at 17:42 +0000, Sun, Xiaodong wrote:
> Hi, Tanu,
> 
> There is about 0.5s constant delay between tunnel sink and local sink
> and these two sinks are never in-sync no matter how long I play. I
> also tried different values for module-combine-sink adjust_time
> parameter and it doesn't help.
> 
> Then, I tried another setup by replacing tunnel sink PC-2 using
> another embedded audio device. The symptom is different. The tunnel
> sink (embedded audio device) lags local sink (PC-1) at the beginning.
> Then after playing a while, these two sinks can be in-sync eventually.
> The period from un-sync to sync differs when different adjust_time is
> applied. The shortest time is 1.5s, when adjust_time is set to 1. I
> traced the logs, and found that during this period, either the latency
> of one of the sinks is too much different from the other or adjusted
> rate of one of the sinks jumps back and forth around the target sample
> rate.   
> 
> I looked at the source code of module-combine-sink.c. The
> adjust_rates() function seems to try to reduce sample rate of the sink
> with smaller latency through resampling module, in order to increase
> that sink's latency and eventually make the latency of both sinks
> same.

This will eventually make the streams synchronized, so it's a good
approach.

> This algorithm chooses to stop doing rate adjustment when the
> difference between sinks are too much different (more than
> ~0.2*adjust_time seconds).

I guess you're referring to this code:

        if (new_rate < (uint32_t) (base_rate*0.8) || new_rate > (uint32_t) (base_rate*1.25)) {
            pa_log_warn("[%s] sample rates too different, not adjusting (%u vs. %u).", o->sink_input->sink->name, base_rate, new_rate);
            new_rate = base_rate;
        }

It may be that whoever wrote that didn't take large initial latency
differences into account, so this sanity check might need some tweaking.

> And also the new adjusted rate can not exceed 0.2% of the current
> rate.

This ensures that it's not audible when the rate changes. This shouldn't
break anything, though - it will only make synchronizing time longer
(and possibly lead to underruns, but that's not necessarily a bad thing,
since if the latency difference is large, it's maybe a good idea to drop
some audio to reach synchronization faster).

> All these logics seem telling me that this rate adjustment algorithm
> is designed to provide synchronizations between sinks with only minor
> latency difference, which are mainly caused by Audio DAC sampling
> clock deviation on different sink devices, not for devices with major
> latency difference, which could be caused by network or buffering in
> audio S/W stack.

Yes, I think it's mainly designed for handling clock deviation issues.
But it shouldn't be unreasonably difficult to change the code to handle
also large initial latency differences.

> For synchronized playback with major latency difference, a common
> timestamp (e.g. PTS in AV sync) should be used. This timestamp is sent
> to all sinks before real playback starts. All sinks can only buffer
> audio sample data, but can not start playback until the local time
> matches timestamp. The local timer synchronization can be achieved by
> either NTP or PTP protocol. Does pulseaudio include such or similar
> scheme for combined sink devices' synchronization playback? 

No, there's no support for that. Synchronized playback start requires
another algorithm (something like what you suggested) in addition to the
algorithm that ensures that the streams stay synchronized even with
deviating clocks.

-- 
Tanu