[pulseaudio-discuss] a question about audio synchronization between local sink and tunnel sink

[Sun, Xiaodong]

Tanu,

Yes, I am new to PulseAudio code. All I have done so far with code is, reading a few module source code (i.e. module-combine-sink.c, module-tunnel.c, sink.c, etc.) to understand the nature of this unsynchronized issue I met. I didn't expect I will be involved as a developer at all. Since this feature is very important to me, it seems I have to dive deeper. My time is very tight though. Is there any design document you can share with me. I can only find the document for developer on freedesktop.org. 

Thanks!

Xiaodong

-----Original Message-----
From: Tanu Kaskinen [mailto:tanuk at iki.fi] 
Sent: Thursday, October 18, 2012 11:45 AM
To: Sun, Xiaodong
Cc: pulseaudio-discuss at lists.freedesktop.org
Subject: Re: [pulseaudio-discuss] a question about audio synchronization between local sink and tunnel sink

On Wed, 2012-10-17 at 18:12 +0000, Sun, Xiaodong wrote:
> Hi, Tanu,
> 
> Yes, with my embedded sink device. I can get synchronized playback 
> eventually. But the time between playing start and synchronization is 
> too long. The minimum is 1.5 minutes (sorry 1.5s is a typo error). So 
> it is not good enough for synchronized playback.
> 
> Now there are two directions to improve this. The first (more feasible 
> but not perfect) is tweaking the adjust_rates() function and make 
> synchronization faster when latency is bigger. The second (don't know 
> how much effort but should be a perfect solution) is adding a new 
> algorithm based on timestamp besides current rate adjust algorithm.
> 
> I will try the first one since it is a feasible method for me to try.
> For the second one, how much effort do you think there is? And do you 
> have any plan to do this kind of improvement?

Answering the second question first: I don't have any plan of implementing synchronized stream start in the foreseeable future. I'm not opposed to having such feature, though, if someone else implements it.

Then, how much effort I think there is... You should start by making it possible to create a stream in a corked (paused) state and schedule its start at an exact point in time. Creating a stream in a corked state is already possible by using the PA_SINK_INPUT_START_CORKED flag (or PA_STREAM_START_CORKED in the public libpulse API). What's missing is a way to request uncorking to a happen exactly at a certain time in the future. Trying to think how to implement the scheduling in practice...
it's not possible to have zero latency, so the playback has to start before the time the first sample is supposed to reach the DAC output. It might make sense to not use the normal corking functionality, because it's unpredictable what the latency will be after the uncorking.
Instead, the sink input could generate silence while waiting for the right time to come. When the sink input starts to play, it will have the exact latency information, so it can calculate the amount of silence that it will need to generate before starting to play the real data.
That doesn't sound too difficult, although if you're new to the code, it can take a fair bit of time to figure out what code to touch and how.
Feel free to ask help in IRC.

The first step could be adding a function to the public API for scheduling stream starting. It could be tied to the corking functionality after all, so clients could create the stream using PA_STREAM_START_CORKED and then request uncorking at a given time.
Behind the scenes, the sink input would already at this point uncork itself and start generating silence.

The new function in the public API could mirror the pa_stream_cork() function. This is the pa_stream_cork() definition:

    pa_operation *pa_stream_cork(pa_stream *s, int b, pa_stream_success_cb_t cb, void *userdata);

The new function could be similar, but with an extra argument:

    pa_operation *pa_stream_cork_at(pa_stream *s, int b, time_t t, pa_stream_success_cb_t cb, void *userdata);

So, that function could also be used for pausing a stream at a given time, which I guess is potentially useful too.

After that's done, module-combine-sink will still need to be modified to take advantage of the new feature.

--
Tanu