[pulseaudio-discuss] Crackling audio with Pulseaudio 4.0 and the simple Pulse API.

[Alexander E. Patrakov]

2013/7/11 Tanu Kaskinen <tanu.kaskinen at linux.intel.com>:
> The first is that I believe simple reset results in some inaccuracy at
> least for the next sample that the resampler produces, because a reset
> means that the resampler pretends that the previous audio was all
> silence, which is not true. That said, we have been doing this forever,
> and nobody has complained, and I would guess that the error is rarely
> audible.

Please count this mail as a complaint, then. The error is audible (as
a click), I guess nobody reported this because in the most common
rewind case (on stream volume adjustment) one simply takes clicks as
something expected.

> The second issue is that if the resampler does any buffering (like the
> wrapper that you perhaps plan to do, or swr_convert()), and the buffered
> data is dropped on rewind, the dropped data should be accounted for in
> the next component in the rewind chain - the dropped data should be sort
> of pushed back to where it originally came from, otherwise that data is
> completely lost, so the audio skips (very little, but perhaps audibly).

I don't quite understand this part. I think that, if a resampler
"eats" the beginning of the incoming audio data, it would have done so
during the previous piece of sound (that existed before the rewind),
too. So the end result would be not that some data is dropped due to a
rewind, but that the boundary between the old and amended data would
be off by some samples. Your third comment about the latency is spot
on. If the resampler allows one to set the initial phase (and that
conflicts with any form of the reset() api) and the amended sound is
the same as the original one, there will be no error.

> The third issue is that if the resampler does any buffering, that
> affects the latency, and that is not currently taken into account in the
> latency calculations. The added latency is perhaps small enough not to
> matter at all.

It is typically less than 1 ms.

OTOH, the above discussion makes me think that the API for resamplers,
as defined in src/pulsecore/resampler.h, is unsuitable if rewinds are
allowed and have to be processed absolutely correctly. Also I think
that the correct API (that allows pulseaudio to tell resamplers
explicitly about any rewinds, so that they adjust the phase
accordingly and access old samples directly as needed) is not
implementable on top of any stock resampler if it is treated as a
black box.

In Wine (where I am the author of the resampling code), the problem of
interaction between resampling and rewinds is solved by basing the
resampler on the "pull", not "push" model - i.e. essentially having a
stateless function that computes the Nth output sample using only N
(for phase information that relates positions of input and output
samples on the time axis) and the whole array of input samples.
Unfortunately, no stock resampler provides this, so that's why I had
to write custom code. And it's rather slow :(

-- 
Alexander E. Patrakov