[pulseaudio-discuss] a question about audio synchronization between local sink and tunnel sink

[Tanu Kaskinen]

On Wed, 2012-10-24 at 23:49 +0000, Sun, Xiaodong wrote:
> Hi, Tanu,
> 
> First of all, thanks for your suggestion. This is definitely the
> simplest way to solve this issue as long as it works as you expect, 
> as adding a new API in PA core also means changing application is
> required.
> 
> I added following lines right before request_memblock(o, nbytes) in
> module-combine-sink.c function sink_input_pop_cb().
> 
>     pa_usec_t sink_input_latency, sink_latency, total_latency;
>     sink_input_latency = pa_sink_input_get_latency_within_thread(i, &sink_latency);
>     total_latency = sink_input_latency + sink_latency;
>     pa_memblockq_seek(o->memblockq, pa_usec_to_bytes(total_latency, &o->sink->sample_spec), PA_SEEK_RELATIVE, TRUE);
> 
> Then I define pa_sink_input_get_latency_within_thread() function in
> sink_input.c
> 
> But the new code gave me an error log message like this when I am
> trying to create a combine sink (combining a local alsa sink and a
> tunnel sink):
> 
> E: [alsa-sink] memblockq.c: Assertion 'length % bq->base == 0' failed at pulsecore/memblockq.c:613, function pa_memblockq_drop(). Aborting.
> Aborted

This might be caused by using o->sink->sample_spec in the
pa_memblockq_seek() call. You should use o->sink_input->sample_spec when
dealing with the output's memblockq. o->sink is the sink that the output
is connected to, and it may have a different sample spec than
o->sink_input, which uses the sample spec of the combine sink. The
output's memblockq contains data in the format of the combine sink.

> Here is my questions:
> 
> 1. Is that seek function call correct?

What the call does is that it moves the write index of the memblockq
forward. That will cause a gap (silence) in the memblockq, which is not
what you want, especially not every time the sink_input_pop_cb() is
called. You should seek in the buffer only the first time the function
is called. By "seek" I don't necessarily mean pa_memblockq_seek() -
rather than moving the write index, you'll need to move the read index.
That's done with pa_memblockq_drop() or pa_memblock_rewind(), depending
on whether you need to move the read index forward or backward.

It's not clear to me what the exact algorithm should be for figuring out
how much to move the read index. I think you need to somehow keep track
of how much in total has been played (meaning how many bytes have come
out of the speakers), I'll call that number the "played_count".

When the first output is created, nothing has come out of the speakers.
When sink_input_pop_cb is called for the first time ever, the
played_count is actually negative, because there is some latency. The
first output doesn't have to do any seeking, since there are no other
outputs to synchronize with.

When the second output is created, sink_input_pop_cb will somehow have
to figure out what the current played_count is, and if the first chunk
in the output's memblockq isn't the first chunk that has ever been gone
through the combine sink, the output needs to know how much data has
there been before the first chunk in the memblockq. I'll call that
number the "skipped_count".

Once you have somehow figured out the current played_count and the
skipped_count, and you have the total_latency of the second output too,
you can calculate how much and to what direction the read index of the
output's memblockq needs to be moved (negative result means moving
backward and positive forward):

    move_amount = played_count - skipped_count + total_latency;

> 2. Following is my understanding of your suggestion, is it true?
>     The new output stream need to seek to a starting point represented
> using latencies (should include its own latency and another output
> stream latency). In my case, there are two output streams (alsa sink
> output and tunnel sink output). Each of them have a separate buffer
> (memblockq ?). But the data of these buffers comes from the same
> virtual sink (called combined). When new output stream has smaller
> latency (latency_1) than the other one (latency_2), then the new
> output stream need to skip to starting point (latency_2 - latency_1)
> when filling its buffer with data from virtual sink. However when new
> output stream has bigger latency than the other one, then the new
> output stream need to fill silence data with size (latency_1 -
> latency_2)*sample_rate*sample_size into its buffer to compensate this
> delay. If my understanding is correct, then the above modification is
> not enough.

Sounds roughly like what I described above. Note that the silence
generation happens automatically. At the beginning the memblockq read
index is 0. If you then move the read index backward
(pa_memblockq_rewind()), the read index becomes negative. The first data
chunk in the buffer starts at index 0, so when data is read from the
memblockq for the first time with pa_memblockq_peek(),
pa_memblockq_peek() will generate silence.

> 3. What is the purpose of sink_input? Why we can not use sink
> directly?

Sinks get their data from sink inputs. I don't know what you mean by
"using sink directly".

-- 
Tanu