[pulseaudio-discuss] a question about audio synchronization between local sink and tunnel sink

[Sun, Xiaodong]

Hi, Tanu,

First of all, thanks for your suggestion. This is definitely the simplest way to solve this issue as long as it works as you expect, 
as adding a new API in PA core also means changing application is required.

I added following lines right before request_memblock(o, nbytes) in module-combine-sink.c function sink_input_pop_cb().

    pa_usec_t sink_input_latency, sink_latency, total_latency;
    sink_input_latency = pa_sink_input_get_latency_within_thread(i, &sink_latency);
    total_latency = sink_input_latency + sink_latency;
    pa_memblockq_seek(o->memblockq, pa_usec_to_bytes(total_latency, &o->sink->sample_spec), PA_SEEK_RELATIVE, TRUE);

Then I define pa_sink_input_get_latency_within_thread() function in sink_input.c

But the new code gave me an error log message like this when I am trying to create a combine sink (combining a local alsa sink and a tunnel sink):

E: [alsa-sink] memblockq.c: Assertion 'length % bq->base == 0' failed at pu  lsecore/memblockq.c:613, function pa_memblockq_drop(). Aborting.
Aborted

Here is my questions:

1. Is that seek function call correct?
2. Following is my understanding of your suggestion, is it true?
    The new output stream need to seek to a starting point represented using latencies (should include its own latency and another output stream latency). In my case, there are two output streams (alsa sink output and tunnel sink output). Each of them have a separate buffer (memblockq ?). But the data of these buffers comes from the same virtual sink (called combined). When new output stream has smaller latency (latency_1) than the other one (latency_2), then the new output stream need to skip to starting point (latency_2 - latency_1) when filling its buffer with data from virtual sink. However when new output stream has bigger latency than the other one, then the new output stream need to fill silence data with size (latency_1 - latency_2)*sample_rate*sample_size into its buffer to compensate this delay. If my understanding is correct, then the above modification is not enough.
3. What is the purpose of sink_input? Why we can not use sink directly?

Thanks!

Xiaodong 

-----Original Message-----
From: Tanu Kaskinen [mailto:tanuk at iki.fi] 
Sent: Wednesday, October 24, 2012 1:27 AM
To: Sun, Xiaodong
Cc: pulseaudio-discuss at lists.freedesktop.org
Subject: Re: [pulseaudio-discuss] a question about audio synchronization between local sink and tunnel sink

On Thu, 2012-10-18 at 21:45 +0300, Tanu Kaskinen wrote:
> On Wed, 2012-10-17 at 18:12 +0000, Sun, Xiaodong wrote:
> > Hi, Tanu,
> > 
> > Yes, with my embedded sink device. I can get synchronized playback 
> > eventually. But the time between playing start and synchronization 
> > is too long. The minimum is 1.5 minutes (sorry 1.5s is a typo 
> > error). So it is not good enough for synchronized playback.
> > 
> > Now there are two directions to improve this. The first (more 
> > feasible but not perfect) is tweaking the adjust_rates() function 
> > and make synchronization faster when latency is bigger. The second 
> > (don't know how much effort but should be a perfect solution) is 
> > adding a new algorithm based on timestamp besides current rate adjust algorithm.
> > 
> > I will try the first one since it is a feasible method for me to try.
> > For the second one, how much effort do you think there is? And do 
> > you have any plan to do this kind of improvement?
> 
> Answering the second question first: I don't have any plan of 
> implementing synchronized stream start in the foreseeable future. I'm 
> not opposed to having such feature, though, if someone else implements 
> it.
> 
> Then, how much effort I think there is... You should start by making 
> it possible to create a stream in a corked (paused) state and schedule 
> its start at an exact point in time.

Actually, I think this is not necessary. I believe module-combine-sink can do near-perfect synchronization by its own, without touching code elsewhere. When module-combine-sink creates a new output stream, it can check the latency of the new stream once the stream is attached to the output sink. If there are other outputs (so the new output stream needs to be synchronized with them), module-combine-sink can seek in the input buffer to an appropriate place when it writes the first chunk of audio to the new stream. That way the new stream will immediately start playing at the same place that other streams are playing.

I don't know how big changes this would require in module-combine-sink, possibly quite big, but at least you don't have to add new functionality to the pulseaudio core.

Here's how to ask the stream latency: in sink_input_pop_cb() in module-combine-sink.c, do this when the sink asks for the first chunk of
audio:

    pa_usec_t sink_input_latency, sink_latency, total_latency;

    sink_input_latency = pa_sink_input_get_latency_within_thread(i, &sink_latency);
    total_latency = sink_input_latency + sink_latency;
    /* then seek in the input buffer as appropriate */

There's a problem, though: pa_sink_input_get_latency_within_thread()
doesn't exist. But you can add it. The code should look like this:

    pa_usec_t pa_sink_input_get_latency_within_thread(pa_sink_input *i, pa_usec_t *sink_latency) {
        pa_sink_input_assert_ref(i);
        pa_sink_input_assert_io_context(i);

        if (sink_latency)
            *sink_latency = pa_sink_get_latency_within_thread(i->sink);

        return pa_bytes_to_usec(pa_memblockq_get_length(i->thread_info.render_memblockq), &i->sink->sample_spec);
    }

--
Tanu