[pulseaudio-commits] [Git][pulseaudio/pulseaudio][master] 10 commits: sink-input: Add history memblockq
PulseAudio Marge Bot (@pulseaudio-merge-bot)
gitlab at gitlab.freedesktop.org
Wed Nov 3 18:40:16 UTC 2021
PulseAudio Marge Bot pushed to branch master at PulseAudio / pulseaudio
Commits:
1bfabd65 by Georg Chini at 2021-11-03T18:37:31+00:00
sink-input: Add history memblockq
A new memblockq is added to the sink input code to keep some history of the
input data. The queue is kept in sync with the render memblockq. The old input
data will be used to prepare the resampler after a rewind.
pa_resampler_request() and pa_resampler_result() have been changed to round
as good as possible to avoid loosing or duplicating samples during rewinds.
Part-of: <https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests/120>
- - - - -
656179a8 by Georg Chini at 2021-11-03T18:37:31+00:00
resampler: Add pa_resampler_prepare() and pa_resampler_get_delay() functions
The pa_resampler_get_delay() function allows to retrieve the current resampler
delay in input samples for all supported resamplers. The return value is a double
to maintain precision when using variable rate resamplers. Because in many places
the delay is needed in usec, pa_resampler_get_delay_usec() was also supplied.
The speex resampler now skips leading zero samples to provide meaningful delay values.
In the next patch, the pa_resampler_prepare() function will be used to train the
resampler after a rewind. It takes data from a history memblockq and runs it through
the resampler. The output data is discarded.
To make this logic possible, the soxr resampler had to be converted to use variable
rate. The fixed rate version has a variable delay, therefore the logic above could
not be applied. Additionally, with fixed rate, the delay is larger than 150ms in
some situations, while with variable rate the delay is fixed and comparable to the
other resamplers.
Part-of: <https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests/120>
- - - - -
da539ed3 by Georg Chini at 2021-11-03T18:37:31+00:00
sink-input: Implement resampler pseudo rewinding
This patch uses the two previous patches to implemnt pseudo-rewinding for the
resamplers by feeding some old data into the resampler after a reset. This is
necessary because PA is using external resamplers that do not implement
rewinding.
To get exactly the same output data from the resampler after a rewind if possible,
the matching period is calculated. This is the number of input samples that produces
an integral number of output samples. After the matching period, the resampler state
repeats. If the matching period is not too large, feeding history into the resampler
will start at a point that is a multiple of the matching period back in time. Then
the resampler will produce exactly the same samples.
The PA_RESAMPLER_MAX_HISTORY value has been replaced by PA_RESAMPLER_MAX_DELAY_USEC
and the required number of history samples is calculated from the sink input sample
rate. The number of history samples can be as large as about 12500.
This fixes glitches during volume changes when the sink runs on a rate different
from the sink input rate.
Part-of: <https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests/120>
- - - - -
a275a0b8 by Georg Chini at 2021-11-03T18:37:31+00:00
sink-input: Change move logic
The introduction of the history queue makes it possible to implement moving
of streams without involving the implementer. Instead of dropping all data
from the render memblockq and requesting the implementer to rewrite the
data, the render memblockq is now reconstructed from the history queue.
Additionally, the render queue will be filled with silence matching the
amount of audio that is left playing on the old sink to avoid playing
the same audio twice.
This patch slightly breaks moving for virtual sinks because they do not
yet include the resampler delay in their latency reports. This will be
fixed in a different patch set.
Part-of: <https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests/120>
- - - - -
d55fde2f by Georg Chini at 2021-11-03T18:37:31+00:00
source-output: Fix rewinding
If the output implements a process_rewind() callback, the resampler delay is
not taken into account. This leads to glitches during volume changes when
source and source output rates differ.
This patch fixes the problem.
Part-of: <https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests/120>
- - - - -
129a59a2 by Georg Chini at 2021-11-03T18:37:31+00:00
virtual sources: Include resampler delay in latency reports
Part-of: <https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests/120>
- - - - -
f121779f by Georg Chini at 2021-11-03T18:37:31+00:00
loopback: Add resampler delay to latency snapshots
Part-of: <https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests/120>
- - - - -
5c6d91a9 by Georg Chini at 2021-11-03T18:37:31+00:00
sink-input: Limit rewinding to max_rewind for virtual sinks
This patch is in preparation of allowing virtual sinks to specify their own
max_rewind limit.
Currently, virtual sinks cannot specify their max_rewind limit, but just copy
the value from the master sink. This may not be correct, if the DSP code of the
virtual sink has limited (or no) rewinding capability.
Because the DSP code of the virtual sink is rewound in the process_rewind()
callback of the sink input, it must be ensured, that rewinding a sink input
to the master of a virtual sink is limited similar to rewinding a sink.
There are two remaining exceptions:
1) If an underrun is detected. In that case, the filter should be reset anyway.
2) When the sink input of the filter is moved and attached to the destination
sink.
The move case is handled without involvement of the implementer, so the implementer
can only receive a rewind larger than max_rewind when the filter should be reset
anyway.
All existing virtual sinks do not distinguish between reset and rewind of the
filter.
Part-of: <https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests/120>
- - - - -
a37fd7ea by Georg Chini at 2021-11-03T18:37:31+00:00
sink-input: Query sink inputs for max_rewind value when setting max_rewind
This patch is in preparation of allowing virtual sinks to specify their own
max_rewind limit.
Currently pa_sink_set_max_rewind_within_thread() simply sets the value of
max_rewind and informs the sink inputs about the new value. Virtual sinks
may however provide their own limit on max_rewind.
This patch allows to query the active sink inputs for the max_rewind value
they support and sets max_rewind to the minimum supported value. This way,
the max_rewind value from the virtual sinks can be communicated to the master
sink.
Part-of: <https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests/120>
- - - - -
851c377d by Georg Chini at 2021-11-03T18:37:31+00:00
tests: Add resampler rewind test
This patch adds a test program that generates a square wave of a given frequency,
length and sample rate. This is then resampled to another rate, rewound and the
rewound part is run through the resampler again. After that, the results of the
first and second resampler pass are compared.
Part-of: <https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests/120>
- - - - -
16 changed files:
- src/.gitignore
- src/modules/echo-cancel/module-echo-cancel.c
- src/modules/module-loopback.c
- src/modules/module-remap-source.c
- src/modules/module-virtual-source.c
- src/pulsecore/resampler.c
- src/pulsecore/resampler.h
- src/pulsecore/resampler/soxr.c
- src/pulsecore/resampler/speex.c
- src/pulsecore/sink-input.c
- src/pulsecore/sink-input.h
- src/pulsecore/sink.c
- src/pulsecore/sink.h
- src/pulsecore/source-output.c
- src/tests/meson.build
- + src/tests/resampler-rewind-test.c
Changes:
=====================================
src/.gitignore
=====================================
@@ -71,6 +71,7 @@ proplist-test
queue-test
remix-test
resampler-test
+resampler-rewind-test
rtpoll-test
rtstutter
sig2str-test
=====================================
src/modules/echo-cancel/module-echo-cancel.c
=====================================
@@ -422,6 +422,9 @@ static int source_process_msg_cb(pa_msgobject *o, int code, void *data, int64_t
/* and the buffering we do on the source */
pa_bytes_to_usec(u->source_output_blocksize, &u->source_output->source->sample_spec);
+ /* Add resampler delay */
+ *((int64_t*) data) += pa_resampler_get_delay_usec(u->source_output->thread_info.resampler);
+
return 0;
case PA_SOURCE_MESSAGE_SET_VOLUME_SYNCED:
=====================================
src/modules/module-loopback.c
=====================================
@@ -567,6 +567,7 @@ static void source_output_push_cb(pa_source_output *o, const pa_memchunk *chunk)
/* Send current source latency and timestamp with the message */
push_time = pa_rtclock_now();
current_source_latency = pa_source_get_latency_within_thread(u->source_output->source, true);
+ current_source_latency += pa_resampler_get_delay_usec(u->source_output->thread_info.resampler);
pa_asyncmsgq_post(u->asyncmsgq, PA_MSGOBJECT(u->sink_input), SINK_INPUT_MESSAGE_POST, PA_INT_TO_PTR(current_source_latency), push_time, chunk, NULL);
u->send_counter += (int64_t) chunk->length;
@@ -599,6 +600,9 @@ static int source_output_process_msg_cb(pa_msgobject *obj, int code, void *data,
/* Add content of delay memblockq to the source latency */
u->latency_snapshot.source_latency = pa_source_get_latency_within_thread(u->source_output->source, true) +
pa_bytes_to_usec(length, &u->source_output->source->sample_spec);
+ /* Add resampler latency */
+ u->latency_snapshot.source_latency += pa_resampler_get_delay_usec(u->source_output->thread_info.resampler);
+
u->latency_snapshot.source_timestamp = pa_rtclock_now();
return 0;
@@ -894,8 +898,9 @@ static int sink_input_process_msg_cb(pa_msgobject *obj, int code, void *data, in
time_delta = PA_PTR_TO_INT(data);
/* Add the time between push and post */
time_delta += pa_rtclock_now() - (pa_usec_t) offset;
- /* Add the sink latency */
+ /* Add the sink and resampler latency */
time_delta += pa_sink_get_latency_within_thread(u->sink_input->sink, true);
+ time_delta += pa_resampler_get_delay_usec(u->sink_input->thread_info.resampler);
/* The source latency report includes the audio in the chunk,
* but since we already pushed the chunk to the memblockq, we need
@@ -972,6 +977,9 @@ static int sink_input_process_msg_cb(pa_msgobject *obj, int code, void *data, in
/* Add content of render memblockq to sink latency */
u->latency_snapshot.sink_latency = pa_sink_get_latency_within_thread(u->sink_input->sink, true) +
pa_bytes_to_usec(length, &u->sink_input->sink->sample_spec);
+ /* Add resampler latency */
+ u->latency_snapshot.sink_latency += pa_resampler_get_delay_usec(u->sink_input->thread_info.resampler);
+
u->latency_snapshot.sink_timestamp = pa_rtclock_now();
return 0;
=====================================
src/modules/module-remap-source.c
=====================================
@@ -101,6 +101,9 @@ static int source_process_msg_cb(pa_msgobject *o, int code, void *data, int64_t
/* Add the latency internal to our source output on top */
pa_bytes_to_usec(pa_memblockq_get_length(u->source_output->thread_info.delay_memblockq), &u->source_output->source->sample_spec);
+ /* Add resampler delay */
+ *((int64_t*) data) += pa_resampler_get_delay_usec(u->source_output->thread_info.resampler);
+
return 0;
}
=====================================
src/modules/module-virtual-source.c
=====================================
@@ -177,6 +177,9 @@ static int source_process_msg_cb(pa_msgobject *o, int code, void *data, int64_t
/* FIXME, no idea what I am doing here */
pa_bytes_to_usec(pa_memblockq_get_length(u->source_output->thread_info.delay_memblockq), &u->source_output->source->sample_spec);
+ /* Add resampler delay */
+ *((int64_t*) data) += pa_resampler_get_delay_usec(u->source_output->thread_info.resampler);
+
return 0;
}
=====================================
src/pulsecore/resampler.c
=====================================
@@ -22,8 +22,10 @@
#endif
#include <string.h>
+#include <math.h>
#include <pulse/xmalloc.h>
+#include <pulse/timeval.h>
#include <pulsecore/log.h>
#include <pulsecore/macro.h>
#include <pulsecore/strbuf.h>
@@ -120,6 +122,24 @@ static int (* const init_table[])(pa_resampler *r) = {
#endif
};
+static void calculate_gcd(pa_resampler *r) {
+ unsigned gcd, n;
+
+ pa_assert(r);
+
+ gcd = r->i_ss.rate;
+ n = r->o_ss.rate;
+
+ while (n != 0) {
+ unsigned tmp = gcd;
+
+ gcd = n;
+ n = tmp % n;
+ }
+
+ r->gcd = gcd;
+}
+
static pa_resample_method_t choose_auto_resampler(pa_resample_flags_t flags) {
pa_resample_method_t method;
@@ -163,9 +183,6 @@ static pa_resample_method_t fix_method(
}
/* Else fall through */
case PA_RESAMPLER_FFMPEG:
- case PA_RESAMPLER_SOXR_MQ:
- case PA_RESAMPLER_SOXR_HQ:
- case PA_RESAMPLER_SOXR_VHQ:
if (flags & PA_RESAMPLER_VARIABLE_RATE) {
pa_log_info("Resampler '%s' cannot do variable rate, reverting to resampler 'auto'.", pa_resample_method_to_string(method));
method = PA_RESAMPLER_AUTO;
@@ -349,10 +366,13 @@ pa_resampler* pa_resampler_new(
r->mempool = pool;
r->method = method;
r->flags = flags;
+ r->in_frames = 0;
+ r->out_frames = 0;
/* Fill sample specs */
r->i_ss = *a;
r->o_ss = *b;
+ calculate_gcd(r);
if (am)
r->i_cm = *am;
@@ -479,7 +499,12 @@ void pa_resampler_set_input_rate(pa_resampler *r, uint32_t rate) {
if (r->i_ss.rate == rate)
return;
+ /* Recalculate delay counters */
+ r->in_frames = pa_resampler_get_delay(r, false);
+ r->out_frames = 0;
+
r->i_ss.rate = rate;
+ calculate_gcd(r);
r->impl.update_rates(r);
}
@@ -492,7 +517,12 @@ void pa_resampler_set_output_rate(pa_resampler *r, uint32_t rate) {
if (r->o_ss.rate == rate)
return;
+ /* Recalculate delay counters */
+ r->in_frames = pa_resampler_get_delay(r, false);
+ r->out_frames = 0;
+
r->o_ss.rate = rate;
+ calculate_gcd(r);
r->impl.update_rates(r);
@@ -500,34 +530,73 @@ void pa_resampler_set_output_rate(pa_resampler *r, uint32_t rate) {
pa_lfe_filter_update_rate(r->lfe_filter, rate);
}
+/* pa_resampler_request() and pa_resampler_result() should be as exact as
+ * possible to ensure that no samples are lost or duplicated during rewinds.
+ * Ignore the leftover buffer, the value appears to be wrong for ffmpeg
+ * and 0 in all other cases. If the resampler is NULL it means that no
+ * resampling is necessary and the input length equals the output length.
+ * FIXME: These functions are not exact for the soxr resamplers because
+ * soxr uses a different algorithm. */
size_t pa_resampler_request(pa_resampler *r, size_t out_length) {
- pa_assert(r);
+ size_t in_length;
- /* Let's round up here to make it more likely that the caller will get at
- * least out_length amount of data from pa_resampler_run().
- *
- * We don't take the leftover into account here. If we did, then it might
- * be in theory possible that this function would return 0 and
- * pa_resampler_run() would also return 0. That could lead to infinite
- * loops. When the leftover is ignored here, such loops would eventually
- * terminate, because the leftover would grow each round, finally
- * surpassing the minimum input threshold of the resampler. */
- return ((((uint64_t) ((out_length + r->o_fz-1) / r->o_fz) * r->i_ss.rate) + r->o_ss.rate-1) / r->o_ss.rate) * r->i_fz;
+ if (!r || out_length == 0)
+ return out_length;
+
+ /* Convert to output frames */
+ out_length = out_length / r->o_fz;
+
+ /* Convert to input frames. The equation matches exactly the
+ * behavior of the used resamplers and will calculate the
+ * minimum number of input frames that are needed to produce
+ * the given number of output frames. */
+ in_length = (out_length - 1) * r->i_ss.rate / r->o_ss.rate + 1;
+
+ /* Convert to input length */
+ return in_length * r->i_fz;
}
size_t pa_resampler_result(pa_resampler *r, size_t in_length) {
- size_t frames;
+ size_t out_length;
- pa_assert(r);
+ if (!r)
+ return in_length;
- /* Let's round up here to ensure that the caller will always allocate big
- * enough output buffer. */
+ /* Convert to intput frames */
+ in_length = in_length / r->i_fz;
- frames = (in_length + r->i_fz - 1) / r->i_fz;
- if (*r->have_leftover)
- frames += r->leftover_buf->length / r->w_fz;
+ /* soxr processes samples in blocks, depending on the ratio.
+ * Therefore samples that do not fit into a block must be
+ * ignored. */
+ if (r->method == PA_RESAMPLER_SOXR_MQ || r->method == PA_RESAMPLER_SOXR_HQ || r->method == PA_RESAMPLER_SOXR_VHQ) {
+ double ratio;
+ size_t block_size;
+ int k;
+
+ ratio = (double)r->i_ss.rate / (double)r->o_ss.rate;
+
+ for (k = 0; k < 7; k++) {
+ if (ratio < pow(2, k + 1))
+ break;
+ }
+ block_size = pow(2, k);
+ in_length = in_length - in_length % block_size;
+ }
+
+ /* Convert to output frames. This matches exactly the algorithm
+ * used by the resamplers except for the soxr resamplers. */
+
+ out_length = in_length * r->o_ss.rate / r->i_ss.rate;
+ if ((double)in_length * (double)r->o_ss.rate / (double)r->i_ss.rate - out_length > 0)
+ out_length++;
+ /* The libsamplerate resamplers return one sample more if the result is integral and the ratio is not integral. */
+ else if (r->method >= PA_RESAMPLER_SRC_SINC_BEST_QUALITY && r->method <= PA_RESAMPLER_SRC_SINC_FASTEST && r->i_ss.rate > r->o_ss.rate && r->i_ss.rate % r->o_ss.rate > 0 && (double)in_length * (double)r->o_ss.rate / (double)r->i_ss.rate - out_length <= 0)
+ out_length++;
+ else if (r->method == PA_RESAMPLER_SRC_ZERO_ORDER_HOLD && r->i_ss.rate > r->o_ss.rate && (double)in_length * (double)r->o_ss.rate / (double)r->i_ss.rate - out_length <= 0)
+ out_length++;
- return (((uint64_t) frames * r->o_ss.rate + r->i_ss.rate - 1) / r->i_ss.rate) * r->o_fz;
+ /* Convert to output length */
+ return out_length * r->o_fz;
}
size_t pa_resampler_max_block_size(pa_resampler *r) {
@@ -586,20 +655,89 @@ void pa_resampler_reset(pa_resampler *r) {
pa_lfe_filter_reset(r->lfe_filter);
*r->have_leftover = false;
+
+ r->in_frames = 0;
+ r->out_frames = 0;
}
-void pa_resampler_rewind(pa_resampler *r, size_t out_frames) {
+/* This function runs amount bytes of data from the history queue through the
+ * resampler and discards the result. The history queue is unchanged after the
+ * call. This is used to preload a resampler after a reset. Returns the number
+ * of frames produced by the resampler. */
+size_t pa_resampler_prepare(pa_resampler *r, pa_memblockq *history_queue, size_t amount) {
+ size_t history_bytes, max_block_size, out_size;
+ int64_t to_run;
+
pa_assert(r);
- /* For now, we don't have any rewindable resamplers, so we just
- reset the resampler instead (and hope that nobody hears the difference). */
- if (r->impl.reset)
+ if (!history_queue || amount == 0)
+ return 0;
+
+ /* Rewind the LFE filter by the amount of history data. */
+ history_bytes = pa_resampler_result(r, amount);
+ if (r->lfe_filter)
+ pa_lfe_filter_rewind(r->lfe_filter, history_bytes);
+
+ pa_memblockq_rewind(history_queue, amount);
+ max_block_size = pa_resampler_max_block_size(r);
+ to_run = amount;
+ out_size = 0;
+
+ while (to_run > 0) {
+ pa_memchunk in_chunk, out_chunk;
+ size_t current;
+
+ current = PA_MIN(to_run, (int64_t) max_block_size);
+
+ /* Get data from memblockq */
+ if (pa_memblockq_peek_fixed_size(history_queue, current, &in_chunk) < 0) {
+ pa_log_warn("Could not read history data for resampler.");
+
+ /* Restore queue to original state and reset resampler */
+ pa_memblockq_drop(history_queue, to_run);
+ pa_resampler_reset(r);
+ return out_size;
+ }
+
+ /* Run the resampler */
+ pa_resampler_run(r, &in_chunk, &out_chunk);
+
+ /* Discard result */
+ if (out_chunk.length != 0) {
+ out_size += out_chunk.length;
+ pa_memblock_unref(out_chunk.memblock);
+ }
+
+ pa_memblock_unref(in_chunk.memblock);
+ pa_memblockq_drop(history_queue, current);
+ to_run -= current;
+ }
+
+ return out_size;
+}
+
+size_t pa_resampler_rewind(pa_resampler *r, size_t out_bytes, pa_memblockq *history_queue, size_t amount) {
+ pa_assert(r);
+
+ /* For now, we don't have any rewindable resamplers, so we just reset
+ * the resampler if we cannot rewind using pa_resampler_prepare(). */
+ if (r->impl.reset && !history_queue)
r->impl.reset(r);
if (r->lfe_filter)
- pa_lfe_filter_rewind(r->lfe_filter, out_frames);
+ pa_lfe_filter_rewind(r->lfe_filter, out_bytes);
- *r->have_leftover = false;
+ if (!history_queue) {
+ *r->have_leftover = false;
+
+ r->in_frames = 0;
+ r->out_frames = 0;
+ }
+
+ if (history_queue && amount > 0)
+ return pa_resampler_prepare(r, history_queue, amount);
+
+ return 0;
}
pa_resample_method_t pa_resampler_get_method(pa_resampler *r) {
@@ -1469,6 +1607,7 @@ void pa_resampler_run(pa_resampler *r, const pa_memchunk *in, pa_memchunk *out)
pa_assert(in->length % r->i_fz == 0);
buf = (pa_memchunk*) in;
+ r->in_frames += buf->length / r->i_fz;
buf = convert_to_work_format(r, buf);
/* Try to save resampling effort: if we have more output channels than
@@ -1487,6 +1626,7 @@ void pa_resampler_run(pa_resampler *r, const pa_memchunk *in, pa_memchunk *out)
if (buf->length) {
buf = convert_from_work_format(r, buf);
*out = *buf;
+ r->out_frames += buf->length / r->o_fz;
if (buf == in)
pa_memblock_ref(buf->memblock);
@@ -1496,6 +1636,47 @@ void pa_resampler_run(pa_resampler *r, const pa_memchunk *in, pa_memchunk *out)
pa_memchunk_reset(out);
}
+/* Get delay in input frames. Some resamplers may have negative delay. */
+double pa_resampler_get_delay(pa_resampler *r, bool allow_negative) {
+ double frames;
+
+ frames = r->out_frames * r->i_ss.rate / r->o_ss.rate;
+ if (frames >= r->in_frames && !allow_negative)
+ return 0;
+ return r->in_frames - frames;
+}
+
+/* Get delay in usec */
+pa_usec_t pa_resampler_get_delay_usec(pa_resampler *r) {
+
+ if (!r)
+ return 0;
+
+ return (pa_usec_t) (pa_resampler_get_delay(r, false) * PA_USEC_PER_SEC / r->i_ss.rate);
+}
+
+/* Get GCD of input and output rate. */
+unsigned pa_resampler_get_gcd(pa_resampler *r) {
+ pa_assert(r);
+
+ return r->gcd;
+}
+
+/* Get maximum resampler history. The resamplers have finite impulse response, so really old
+ * data (more than 2x the resampler latency) cannot affect the output. This means, that in an
+ * ideal case, we should re-run 2 - 3 times the resampler delay through the resampler when it
+ * is rewound. On the other hand this would mean for high sample rates that more than 25000
+ * samples would need to be used (384k * 33ms). Therefore limit the history to 1.5 times the
+ * maximum resampler delay, which should be fully sufficient in most cases and allows to run
+ * at least more than one delay through the resampler in case of high rates. */
+size_t pa_resampler_get_max_history(pa_resampler *r) {
+
+ if (!r)
+ return 0;
+
+ return (uint64_t) PA_RESAMPLER_MAX_DELAY_USEC * r->i_ss.rate * 3 / PA_USEC_PER_SEC / 2;
+}
+
/*** copy (noop) implementation ***/
static int copy_init(pa_resampler *r) {
=====================================
src/pulsecore/resampler.h
=====================================
@@ -73,6 +73,12 @@ typedef enum pa_resample_flags {
PA_RESAMPLER_CONSUME_LFE = 0x0040U,
} pa_resample_flags_t;
+/* Currently, the soxr reampler has the largest delay of all supported resamplers.
+ * The maximum value below has been obtained empirically and contains a safety
+ * margin of about 3ms. If the resampler configuration is changed or additional
+ * resamplers are added, the constant must be re-evaluated. */
+#define PA_RESAMPLER_MAX_DELAY_USEC 33000
+
struct pa_resampler {
pa_resample_method_t method;
pa_resample_flags_t flags;
@@ -109,6 +115,10 @@ struct pa_resampler {
pa_remap_t remap;
bool map_required;
+ double in_frames;
+ double out_frames;
+ unsigned gcd;
+
pa_lfe_filter_t *lfe_filter;
pa_resampler_impl impl;
@@ -147,8 +157,11 @@ void pa_resampler_set_output_rate(pa_resampler *r, uint32_t rate);
/* Reinitialize state of the resampler, possibly due to seeking or other discontinuities */
void pa_resampler_reset(pa_resampler *r);
+/* Prepare resampler for use by running some old data through it. */
+size_t pa_resampler_prepare(pa_resampler *r, pa_memblockq *history_queue, size_t amount);
+
/* Rewind resampler */
-void pa_resampler_rewind(pa_resampler *r, size_t out_frames);
+size_t pa_resampler_rewind(pa_resampler *r, size_t out_bytes, pa_memblockq *history_queue, size_t amount);
/* Return the resampling method of the resampler object */
pa_resample_method_t pa_resampler_get_method(pa_resampler *r);
@@ -162,6 +175,18 @@ const char *pa_resample_method_to_string(pa_resample_method_t m);
/* Return 1 when the specified resampling method is supported */
int pa_resample_method_supported(pa_resample_method_t m);
+/* Get delay of the resampler in input frames */
+double pa_resampler_get_delay(pa_resampler *r, bool allow_negative);
+
+/* Get delay of the resampler in usec */
+pa_usec_t pa_resampler_get_delay_usec(pa_resampler *r);
+
+/* Get the GCD of input and outpu rate */
+unsigned pa_resampler_get_gcd(pa_resampler *r);
+
+/* Get maximum number of history frames */
+size_t pa_resampler_get_max_history(pa_resampler *r);
+
const pa_channel_map* pa_resampler_input_channel_map(pa_resampler *r);
const pa_sample_spec* pa_resampler_input_sample_spec(pa_resampler *r);
const pa_channel_map* pa_resampler_output_channel_map(pa_resampler *r);
=====================================
src/pulsecore/resampler/soxr.c
=====================================
@@ -65,9 +65,14 @@ static void resampler_soxr_free(pa_resampler *r) {
static void resampler_soxr_reset(pa_resampler *r) {
#if SOXR_THIS_VERSION >= SOXR_VERSION(0, 1, 2)
+ double ratio;
+
pa_assert(r);
soxr_clear(r->impl.data);
+
+ ratio = (double)r->i_ss.rate / (double)r->o_ss.rate;
+ soxr_set_io_ratio(r->impl.data, ratio, 0);
#else
/* With libsoxr prior to 0.1.2 soxr_clear() makes soxr_process() crash afterwards,
* so don't use this function and re-create the context instead. */
@@ -89,23 +94,12 @@ static void resampler_soxr_reset(pa_resampler *r) {
}
static void resampler_soxr_update_rates(pa_resampler *r) {
- soxr_t old_state;
+ double ratio;
pa_assert(r);
- /* There is no update method in libsoxr,
- * so just re-create the resampler context */
-
- old_state = r->impl.data;
- r->impl.data = NULL;
-
- if (pa_resampler_soxr_init(r) == 0) {
- if (old_state)
- soxr_delete(old_state);
- } else {
- r->impl.data = old_state;
- pa_log_error("Failed to update libsoxr sample rates");
- }
+ ratio = (double)r->i_ss.rate / (double)r->o_ss.rate;
+ soxr_set_io_ratio(r->impl.data, ratio, 0);
}
int pa_resampler_soxr_init(pa_resampler *r) {
@@ -116,6 +110,7 @@ int pa_resampler_soxr_init(pa_resampler *r) {
unsigned long quality_recipe;
soxr_quality_spec_t quality;
soxr_error_t err = NULL;
+ double ratio;
pa_assert(r);
@@ -150,14 +145,18 @@ int pa_resampler_soxr_init(pa_resampler *r) {
pa_assert_not_reached();
}
- quality = soxr_quality_spec(quality_recipe, 0);
+ quality = soxr_quality_spec(quality_recipe, SOXR_VR);
- state = soxr_create(r->i_ss.rate, r->o_ss.rate, r->work_channels, &err, &io_spec, &quality, &runtime_spec);
+ /* Maximum resample ratio is 100:1 */
+ state = soxr_create(100, 1, r->work_channels, &err, &io_spec, &quality, &runtime_spec);
if (!state) {
pa_log_error("Failed to create libsoxr resampler context: %s.", (err ? err : "[unknown error]"));
return -1;
}
+ ratio = (double)r->i_ss.rate / (double)r->o_ss.rate;
+ soxr_set_io_ratio(state, ratio, 0);
+
r->impl.free = resampler_soxr_free;
r->impl.reset = resampler_soxr_reset;
r->impl.update_rates = resampler_soxr_update_rates;
=====================================
src/pulsecore/resampler/speex.c
=====================================
@@ -132,6 +132,7 @@ static void speex_reset(pa_resampler *r) {
state = r->impl.data;
pa_assert_se(speex_resampler_reset_mem(state) == 0);
+ speex_resampler_skip_zeros(state);
}
static void speex_free(pa_resampler *r) {
@@ -172,6 +173,8 @@ int pa_resampler_speex_init(pa_resampler *r) {
if (!(state = speex_resampler_init(r->work_channels, r->i_ss.rate, r->o_ss.rate, q, &err)))
return -1;
+ speex_resampler_skip_zeros(state);
+
r->impl.data = state;
return 0;
=====================================
src/pulsecore/sink-input.c
=====================================
@@ -29,6 +29,7 @@
#include <pulse/xmalloc.h>
#include <pulse/util.h>
#include <pulse/internal.h>
+#include <pulse/timeval.h>
#include <pulsecore/core-format.h>
#include <pulsecore/mix.h>
@@ -53,6 +54,66 @@ struct volume_factor_entry {
pa_cvolume volume;
};
+/* Calculate number of input samples for the resampler so that either the number
+ * of input samples or the number of output samples matches the defined history
+ * length. */
+static size_t calculate_resampler_history_bytes(pa_sink_input *i, size_t in_rewind_frames) {
+ size_t history_frames, history_max, matching_period, total_frames, remainder;
+ double delay;
+ pa_resampler *r;
+
+ if (!(r = i->thread_info.resampler))
+ return 0;
+
+ /* Initialize some variables, cut off full seconds from the rewind */
+ total_frames = 0;
+ in_rewind_frames = in_rewind_frames % r->i_ss.rate;
+ history_max = pa_resampler_get_max_history(r);
+
+ /* Get the current internal delay of the resampler */
+ delay = pa_resampler_get_delay(r, false);
+
+ /* Calculate the matching period */
+ matching_period = r->i_ss.rate / pa_resampler_get_gcd(r);
+ pa_log_debug("Integral period length is %lu input frames", matching_period);
+
+ /* If the delay is larger than the length of the history queue, we can only
+ * replay as much as we have. */
+ if ((size_t)delay >= history_max) {
+ history_frames = history_max;
+ pa_log_debug("Resampler delay exceeds maximum history");
+ return history_frames * r->i_fz;
+ }
+
+ /* Initially set the history to 3 times the resampler delay. Use at least 2 ms.
+ * We try to find a value between 2 and 3 times the resampler delay to ensure
+ * that the old data has no impact anymore. See also comment to
+ * pa_resampler_get_max_history() in resampler.c. */
+ history_frames = (size_t)(delay * 3.0);
+ history_frames = PA_MAX(history_frames, r->i_ss.rate / 500);
+
+ /* Check how the rewind fits into multiples of the matching period. */
+ remainder = (in_rewind_frames + history_frames) % matching_period;
+
+ /* If possible, use between 2 and 3 times the resampler delay */
+ if (remainder < (size_t)delay && history_frames - remainder <= history_max)
+ total_frames = in_rewind_frames + history_frames - remainder;
+ /* Else, try above 3 times the delay */
+ else if (history_frames + matching_period - remainder <= history_max)
+ total_frames = in_rewind_frames + history_frames + matching_period - remainder;
+
+ if (total_frames != 0)
+ /* We found a perfect match. */
+ history_frames = total_frames - in_rewind_frames;
+ else {
+ /* Try to use 2.5 times the delay. */
+ history_frames = PA_MIN((size_t)(delay * 2.5), history_max);
+ pa_log_debug("No usable integral matching period");
+ }
+
+ return history_frames * r->i_fz;
+}
+
static struct volume_factor_entry *volume_factor_entry_new(const char *key, const pa_cvolume *volume) {
struct volume_factor_entry *entry;
@@ -286,6 +347,7 @@ static void reset_callbacks(pa_sink_input *i) {
i->send_event = NULL;
i->volume_changed = NULL;
i->mute_changed = NULL;
+ i->get_max_rewind_limit = NULL;
}
/* Called from main context */
@@ -301,6 +363,7 @@ int pa_sink_input_new(
int r;
char *pt;
char *memblockq_name;
+ pa_memchunk silence;
pa_assert(_i);
pa_assert(core);
@@ -562,6 +625,11 @@ int pa_sink_input_new(
i->thread_info.underrun_for_sink = 0;
i->thread_info.playing_for = 0;
i->thread_info.direct_outputs = pa_hashmap_new(pa_idxset_trivial_hash_func, pa_idxset_trivial_compare_func);
+ i->thread_info.move_start_time = 0;
+ i->thread_info.resampler_delay_frames = 0;
+ i->thread_info.origin_sink_latency = 0;
+ i->thread_info.dont_rewrite = false;
+ i->origin_rewind_bytes = 0;
pa_assert_se(pa_idxset_put(core->sink_inputs, i, &i->index) == 0);
pa_assert_se(pa_idxset_put(i->sink->inputs, pa_sink_input_ref(i), NULL) == 0);
@@ -582,6 +650,21 @@ int pa_sink_input_new(
&i->sink->silence);
pa_xfree(memblockq_name);
+ memblockq_name = pa_sprintf_malloc("sink input history memblockq [%u]", i->index);
+ pa_sink_input_get_silence(i, &silence);
+ i->thread_info.history_memblockq = pa_memblockq_new(
+ memblockq_name,
+ 0,
+ MEMBLOCKQ_MAXLENGTH,
+ 0,
+ &i->sample_spec,
+ 0,
+ 1,
+ 0,
+ &silence);
+ pa_xfree(memblockq_name);
+ pa_memblock_unref(silence.memblock);
+
pt = pa_proplist_to_string_sep(i->proplist, "\n ");
pa_log_info("Created input %u \"%s\" on %s with sample spec %s and channel map %s\n %s",
i->index,
@@ -765,6 +848,9 @@ static void sink_input_free(pa_object *o) {
if (i->thread_info.render_memblockq)
pa_memblockq_free(i->thread_info.render_memblockq);
+ if (i->thread_info.history_memblockq)
+ pa_memblockq_free(i->thread_info.history_memblockq);
+
if (i->thread_info.resampler)
pa_resampler_free(i->thread_info.resampler);
@@ -934,6 +1020,7 @@ void pa_sink_input_peek(pa_sink_input *i, size_t slength /* in sink bytes */, pa
* data, so let's just hand out silence */
pa_memblockq_seek(i->thread_info.render_memblockq, (int64_t) slength, PA_SEEK_RELATIVE, true);
+ pa_memblockq_seek(i->thread_info.history_memblockq, (int64_t) ilength_full, PA_SEEK_RELATIVE, true);
i->thread_info.playing_for = 0;
if (i->thread_info.underrun_for != (uint64_t) -1) {
i->thread_info.underrun_for += ilength_full;
@@ -981,6 +1068,9 @@ void pa_sink_input_peek(pa_sink_input *i, size_t slength /* in sink bytes */, pa
pa_volume_memchunk(&wchunk, &i->thread_info.sample_spec, &i->thread_info.soft_volume);
}
+ /* Push chunk into history queue to retain some resampler input history. */
+ pa_memblockq_push(i->thread_info.history_memblockq, &wchunk);
+
if (!i->thread_info.resampler) {
if (nvfs) {
@@ -1045,6 +1135,7 @@ void pa_sink_input_peek(pa_sink_input *i, size_t slength /* in sink bytes */, pa
/* Called from thread context */
void pa_sink_input_drop(pa_sink_input *i, size_t nbytes /* in sink sample spec */) {
+ int64_t rbq, hbq;
pa_sink_input_assert_ref(i);
pa_sink_input_assert_io_context(i);
@@ -1057,6 +1148,22 @@ void pa_sink_input_drop(pa_sink_input *i, size_t nbytes /* in sink sample spec *
#endif
pa_memblockq_drop(i->thread_info.render_memblockq, nbytes);
+
+ /* Keep memblockq's in sync. Using pa_resampler_request()
+ * on nbytes will not work here because of rounding. */
+ rbq = pa_memblockq_get_write_index(i->thread_info.render_memblockq);
+ rbq -= pa_memblockq_get_read_index(i->thread_info.render_memblockq);
+ hbq = pa_memblockq_get_write_index(i->thread_info.history_memblockq);
+ hbq -= pa_memblockq_get_read_index(i->thread_info.history_memblockq);
+ if (rbq >= 0)
+ rbq = pa_resampler_request(i->thread_info.resampler, rbq);
+ else
+ rbq = - (int64_t) pa_resampler_request(i->thread_info.resampler, - rbq);
+
+ if (hbq > rbq)
+ pa_memblockq_drop(i->thread_info.history_memblockq, hbq - rbq);
+ else if (rbq > hbq)
+ pa_memblockq_rewind(i->thread_info.history_memblockq, rbq - hbq);
}
/* Called from thread context */
@@ -1070,6 +1177,7 @@ bool pa_sink_input_process_underrun(pa_sink_input *i) {
if (i->process_underrun && i->process_underrun(i)) {
/* All valid data has been played back, so we can empty this queue. */
pa_memblockq_silence(i->thread_info.render_memblockq);
+ pa_memblockq_silence(i->thread_info.history_memblockq);
return true;
}
return false;
@@ -1079,6 +1187,7 @@ bool pa_sink_input_process_underrun(pa_sink_input *i) {
void pa_sink_input_process_rewind(pa_sink_input *i, size_t nbytes /* in sink sample spec */) {
size_t lbq;
bool called = false;
+ size_t sink_input_nbytes;
pa_sink_input_assert_ref(i);
pa_sink_input_assert_io_context(i);
@@ -1090,21 +1199,27 @@ void pa_sink_input_process_rewind(pa_sink_input *i, size_t nbytes /* in sink sam
#endif
lbq = pa_memblockq_get_length(i->thread_info.render_memblockq);
+ sink_input_nbytes = pa_resampler_request(i->thread_info.resampler, nbytes);
if (nbytes > 0 && !i->thread_info.dont_rewind_render) {
pa_log_debug("Have to rewind %lu bytes on render memblockq.", (unsigned long) nbytes);
pa_memblockq_rewind(i->thread_info.render_memblockq, nbytes);
+ pa_memblockq_rewind(i->thread_info.history_memblockq, sink_input_nbytes);
}
+ if (i->thread_info.dont_rewrite)
+ goto finish;
+
if (i->thread_info.rewrite_nbytes == (size_t) -1) {
/* We were asked to drop all buffered data, and rerequest new
* data from implementor the next time peek() is called */
pa_memblockq_flush_write(i->thread_info.render_memblockq, true);
+ pa_memblockq_flush_write(i->thread_info.history_memblockq, true);
} else if (i->thread_info.rewrite_nbytes > 0) {
- size_t max_rewrite, amount;
+ size_t max_rewrite, sink_amount, sink_input_amount;
/* Calculate how much make sense to rewrite at most */
max_rewrite = nbytes;
@@ -1112,41 +1227,68 @@ void pa_sink_input_process_rewind(pa_sink_input *i, size_t nbytes /* in sink sam
max_rewrite += lbq;
/* Transform into local domain */
- if (i->thread_info.resampler)
- max_rewrite = pa_resampler_request(i->thread_info.resampler, max_rewrite);
+ sink_input_amount = pa_resampler_request(i->thread_info.resampler, max_rewrite);
/* Calculate how much of the rewinded data should actually be rewritten */
- amount = PA_MIN(i->thread_info.rewrite_nbytes, max_rewrite);
+ sink_input_amount = PA_MIN(i->thread_info.rewrite_nbytes, sink_input_amount);
+
+ /* Transform to sink domain */
+ sink_amount = pa_resampler_result(i->thread_info.resampler, sink_input_amount);
- if (amount > 0) {
- pa_log_debug("Have to rewind %lu bytes on implementor.", (unsigned long) amount);
+ if (sink_input_amount > 0) {
+ pa_log_debug("Have to rewind %lu bytes on implementor.", (unsigned long) sink_input_amount);
/* Tell the implementor */
if (i->process_rewind)
- i->process_rewind(i, amount);
+ i->process_rewind(i, sink_input_amount);
called = true;
- /* Convert back to sink domain */
- if (i->thread_info.resampler)
- amount = pa_resampler_result(i->thread_info.resampler, amount);
+ /* Update the write pointer. Use pa_resampler_result(r, sink_input_amount) instead
+ * of sink_amount because the two may differ and the actual replay of the samples
+ * will produce pa_resampler_result(r, sink_input_amount) samples. */
+ pa_memblockq_seek(i->thread_info.render_memblockq, - ((int64_t) pa_resampler_result(i->thread_info.resampler, sink_input_amount)),PA_SEEK_RELATIVE, true);
+
+ /* Rewind the resampler */
+ if (i->thread_info.resampler) {
+ size_t history_bytes;
+ int64_t history_result;
+
+ history_bytes = calculate_resampler_history_bytes(i, sink_input_amount / pa_frame_size(&i->sample_spec));
+
+ if (history_bytes > 0) {
+ history_result = pa_resampler_rewind(i->thread_info.resampler, sink_amount, i->thread_info.history_memblockq, history_bytes);
+
+ /* We may have produced one sample too much or or one sample less than expected.
+ * The replay of the rewound sink input data will then produce a deviation in
+ * the other direction, so that the total number of produced samples matches
+ * pa_resampler_result(r, sink_input_amount + history_bytes). Therefore we have
+ * to correct the write pointer of the render queue accordingly.
+ * Strictly this is only true, if the history can be replayed from a known
+ * resampler state, that is if a true matching period exists. In case where
+ * we are using an approximate matching period, we may still loose or duplicate
+ * one sample during rewind. */
+ history_result -= (int64_t) pa_resampler_result(i->thread_info.resampler, history_bytes);
+ if (history_result != 0)
+ pa_memblockq_seek(i->thread_info.render_memblockq, history_result, PA_SEEK_RELATIVE, true);
+ }
+ }
- if (amount > 0)
- /* Ok, now update the write pointer */
- pa_memblockq_seek(i->thread_info.render_memblockq, - ((int64_t) amount), PA_SEEK_RELATIVE, true);
+ /* Update the history write pointer */
+ pa_memblockq_seek(i->thread_info.history_memblockq, - ((int64_t) sink_input_amount), PA_SEEK_RELATIVE, true);
- if (i->thread_info.rewrite_flush)
+ if (i->thread_info.rewrite_flush) {
pa_memblockq_silence(i->thread_info.render_memblockq);
-
- /* And rewind the resampler */
- if (i->thread_info.resampler)
- pa_resampler_rewind(i->thread_info.resampler, amount);
+ pa_memblockq_silence(i->thread_info.history_memblockq);
+ }
}
}
+finish:
if (!called)
if (i->process_rewind)
i->process_rewind(i, 0);
+ i->thread_info.dont_rewrite = false;
i->thread_info.rewrite_nbytes = 0;
i->thread_info.rewrite_flush = false;
i->thread_info.dont_rewind_render = false;
@@ -1157,7 +1299,7 @@ size_t pa_sink_input_get_max_rewind(pa_sink_input *i) {
pa_sink_input_assert_ref(i);
pa_sink_input_assert_io_context(i);
- return i->thread_info.resampler ? pa_resampler_request(i->thread_info.resampler, i->sink->thread_info.max_rewind) : i->sink->thread_info.max_rewind;
+ return pa_resampler_request(i->thread_info.resampler, i->sink->thread_info.max_rewind);
}
/* Called from thread context */
@@ -1168,11 +1310,14 @@ size_t pa_sink_input_get_max_request(pa_sink_input *i) {
/* We're not verifying the status here, to allow this to be called
* in the state change handler between _INIT and _RUNNING */
- return i->thread_info.resampler ? pa_resampler_request(i->thread_info.resampler, i->sink->thread_info.max_request) : i->sink->thread_info.max_request;
+ return pa_resampler_request(i->thread_info.resampler, i->sink->thread_info.max_request);
}
/* Called from thread context */
void pa_sink_input_update_max_rewind(pa_sink_input *i, size_t nbytes /* in the sink's sample spec */) {
+ size_t max_rewind;
+ size_t resampler_history;
+
pa_sink_input_assert_ref(i);
pa_sink_input_assert_io_context(i);
pa_assert(PA_SINK_INPUT_IS_LINKED(i->thread_info.state));
@@ -1180,8 +1325,15 @@ void pa_sink_input_update_max_rewind(pa_sink_input *i, size_t nbytes /* in the
pa_memblockq_set_maxrewind(i->thread_info.render_memblockq, nbytes);
+ max_rewind = pa_resampler_request(i->thread_info.resampler, nbytes);
+ /* Calculate maximum history needed */
+ resampler_history = pa_resampler_get_max_history(i->thread_info.resampler);
+ resampler_history *= pa_frame_size(&i->sample_spec);
+
+ pa_memblockq_set_maxrewind(i->thread_info.history_memblockq, max_rewind + resampler_history);
+
if (i->update_max_rewind)
- i->update_max_rewind(i, i->thread_info.resampler ? pa_resampler_request(i->thread_info.resampler, nbytes) : nbytes);
+ i->update_max_rewind(i, max_rewind);
}
/* Called from thread context */
@@ -1192,7 +1344,7 @@ void pa_sink_input_update_max_request(pa_sink_input *i, size_t nbytes /* in the
pa_assert(pa_frame_aligned(nbytes, &i->sink->sample_spec));
if (i->update_max_request)
- i->update_max_request(i, i->thread_info.resampler ? pa_resampler_request(i->thread_info.resampler, nbytes) : nbytes);
+ i->update_max_request(i, pa_resampler_request(i->thread_info.resampler, nbytes));
}
/* Called from thread context */
@@ -1754,6 +1906,11 @@ int pa_sink_input_start_move(pa_sink_input *i) {
pa_cvolume_remap(&i->volume_factor_sink, &i->sink->channel_map, &i->channel_map);
+ /* Calculate how much of the latency was rewound on the old sink */
+ i->origin_rewind_bytes = pa_sink_get_last_rewind(i->sink) / pa_frame_size(&i->sink->sample_spec);
+ i->origin_rewind_bytes = i->origin_rewind_bytes * i->sample_spec.rate / i->sink->sample_spec.rate;
+ i->origin_rewind_bytes *= pa_frame_size(&i->sample_spec);
+
i->sink = NULL;
i->sink_requested_by_application = false;
@@ -1904,6 +2061,89 @@ static void set_preferred_sink(pa_sink_input *i, const char *sink_name) {
pa_hook_fire(&i->core->hooks[PA_CORE_HOOK_SINK_INPUT_PREFERRED_SINK_CHANGED], i);
}
+/* Restores the render memblockq from the history memblockq during a move.
+ * Called from main context while the sink input is detached. */
+static void restore_render_memblockq(pa_sink_input *i) {
+ size_t block_size, to_push;
+ size_t latency_bytes = 0;
+ size_t bytes_on_origin_sink = 0;
+ size_t resampler_delay_bytes = 0;
+
+ /* Calculate how much of the latency was left on the old sink */
+ latency_bytes = pa_usec_to_bytes(i->thread_info.origin_sink_latency, &i->sample_spec);
+ if (latency_bytes > i->origin_rewind_bytes)
+ bytes_on_origin_sink = latency_bytes - i->origin_rewind_bytes;
+
+ /* Get resampler latency of old resampler */
+ resampler_delay_bytes = i->thread_info.resampler_delay_frames * pa_frame_size(&i->sample_spec);
+
+ /* Flush the render memblockq and reset the resampler */
+ pa_memblockq_flush_write(i->thread_info.render_memblockq, true);
+ if (i->thread_info.resampler)
+ pa_resampler_reset(i->thread_info.resampler);
+
+ /* Rewind the history queue */
+ if (i->origin_rewind_bytes + resampler_delay_bytes > 0)
+ pa_memblockq_rewind(i->thread_info.history_memblockq, i->origin_rewind_bytes + resampler_delay_bytes);
+
+ /* If something is left playing on the origin sink, add silence to the render memblockq */
+ if (bytes_on_origin_sink > 0) {
+ pa_memchunk chunk;;
+
+ chunk.length = pa_resampler_result(i->thread_info.resampler, bytes_on_origin_sink);
+ if (chunk.length > 0) {
+ chunk.memblock = pa_memblock_new(i->core->mempool, chunk.length);
+ chunk.index = 0;
+ pa_silence_memchunk(&chunk, &i->sink->sample_spec);
+ pa_memblockq_push(i->thread_info.render_memblockq, &chunk);
+ pa_memblock_unref(chunk.memblock);
+ }
+ }
+
+ /* Determine maximum block size */
+ if (i->thread_info.resampler)
+ block_size = pa_resampler_max_block_size(i->thread_info.resampler);
+ else
+ block_size = pa_frame_align(pa_mempool_block_size_max(i->core->mempool), &i->sample_spec);
+
+ /* Now push all the data in the history queue into the render memblockq */
+ to_push = pa_memblockq_get_length(i->thread_info.history_memblockq);
+ while (to_push > 0) {
+ pa_memchunk in_chunk, out_chunk;
+ size_t push_bytes;
+
+ push_bytes = block_size;
+ if (to_push < block_size)
+ push_bytes = to_push;
+
+ if (pa_memblockq_peek_fixed_size(i->thread_info.history_memblockq, push_bytes, &in_chunk) < 0) {
+ pa_log_warn("Could not restore memblockq during move");
+ break;
+ }
+
+ if (i->thread_info.resampler) {
+ pa_resampler_run(i->thread_info.resampler, &in_chunk, &out_chunk);
+ pa_memblock_unref(in_chunk.memblock);
+ } else
+ out_chunk = in_chunk;
+
+ if (out_chunk.length > 0) {
+ pa_memblockq_push(i->thread_info.render_memblockq, &out_chunk);
+ pa_memblock_unref(out_chunk.memblock);
+ }
+
+ pa_memblockq_drop(i->thread_info.history_memblockq, push_bytes);
+ to_push -= push_bytes;
+ }
+
+ /* No need to rewind the history queue here, it will be re-synchronized
+ * with the render queue during the next pa_sink_input_drop() call. */
+
+ /* Tell the sink input not to ask the implementer to rewrite during the
+ * the next rewind */
+ i->thread_info.dont_rewrite = true;
+}
+
/* Called from main context */
int pa_sink_input_finish_move(pa_sink_input *i, pa_sink *dest, bool save) {
struct volume_factor_entry *v;
@@ -1962,7 +2202,9 @@ int pa_sink_input_finish_move(pa_sink_input *i, pa_sink *dest, bool save) {
if (i->state == PA_SINK_INPUT_CORKED)
i->sink->n_corked++;
- pa_sink_input_update_resampler(i);
+ pa_sink_input_update_resampler(i, false);
+
+ restore_render_memblockq(i);
pa_sink_update_status(dest);
@@ -1973,6 +2215,9 @@ int pa_sink_input_finish_move(pa_sink_input *i, pa_sink *dest, bool save) {
pa_assert_se(pa_asyncmsgq_send(i->sink->asyncmsgq, PA_MSGOBJECT(i->sink), PA_SINK_MESSAGE_FINISH_MOVE, i, 0, NULL) == 0);
+ /* Reset move variable */
+ i->origin_rewind_bytes = 0;
+
pa_log_debug("Successfully moved sink input %i to %s.", i->index, dest->name);
/* Notify everyone */
@@ -2112,6 +2357,7 @@ int pa_sink_input_process_msg(pa_msgobject *o, int code, void *userdata, int64_t
pa_usec_t *r = userdata;
r[0] += pa_bytes_to_usec(pa_memblockq_get_length(i->thread_info.render_memblockq), &i->sink->sample_spec);
+ r[0] += pa_resampler_get_delay_usec(i->thread_info.resampler);
r[1] += pa_sink_get_latency_within_thread(i->sink, false);
return 0;
@@ -2216,14 +2462,31 @@ void pa_sink_input_request_rewind(
/* Check if rewinding for the maximum is requested, and if so, fix up */
if (nbytes <= 0) {
- /* Calculate maximum number of bytes that could be rewound in theory */
- nbytes = i->sink->thread_info.max_rewind + lbq;
+ /* Calculate maximum number of bytes that could be rewound in theory.
+ * If the sink has a virtual sink attached, limit rewinding to max_rewind.
+ *
+ * The max_rewind value of a virtual sink depends on the rewinding capability
+ * of its DSP code. The DSP code is rewound in the process_rewind() callback
+ * of the sink input. Therefore rewinding must be limited to max_rewind here. */
+ nbytes = i->sink->thread_info.max_rewind;
+ if (!pa_sink_has_filter_attached(i->sink) && !pa_sink_is_filter(i->sink))
+ nbytes += lbq;
/* Transform from sink domain */
- if (i->thread_info.resampler)
- nbytes = pa_resampler_request(i->thread_info.resampler, nbytes);
+ nbytes = pa_resampler_request(i->thread_info.resampler, nbytes);
}
+ /* For virtual sinks there are two situations where nbytes may exceed max_rewind:
+ * 1) If an underrun was detected.
+ * 2) When the sink input is rewound during a move when it is attached to
+ * the destination sink.
+ * Moving a sink input is handled without involving the implementer, so the
+ * implementer will only be asked to rewind more than max_rewind if an
+ * underrun occurs. In that case, the DSP code of virtual sinks should be
+ * reset instead of rewound. Therefore the rewind function of filters should
+ * check if the requested rewind exceeds the maximum possible rewind of the
+ * filter. */
+
/* Remember how much we actually want to rewrite */
if (i->thread_info.rewrite_nbytes != (size_t) -1) {
if (rewrite) {
@@ -2247,8 +2510,7 @@ void pa_sink_input_request_rewind(
if (nbytes != (size_t) -1) {
/* Transform to sink domain */
- if (i->thread_info.resampler)
- nbytes = pa_resampler_result(i->thread_info.resampler, nbytes);
+ nbytes = pa_resampler_result(i->thread_info.resampler, nbytes);
if (nbytes > lbq)
pa_sink_request_rewind(i->sink, nbytes - lbq);
@@ -2308,7 +2570,7 @@ finish:
/* Called from main context */
/* Updates the sink input's resampler with whatever the current sink requires
* -- useful when the underlying sink's sample spec might have changed */
-int pa_sink_input_update_resampler(pa_sink_input *i) {
+int pa_sink_input_update_resampler(pa_sink_input *i, bool flush_history) {
pa_resampler *new_resampler;
char *memblockq_name;
@@ -2345,6 +2607,9 @@ int pa_sink_input_update_resampler(pa_sink_input *i) {
} else
new_resampler = NULL;
+ if (flush_history)
+ pa_memblockq_flush_write(i->thread_info.history_memblockq, true);
+
if (new_resampler == i->thread_info.resampler)
return 0;
=====================================
src/pulsecore/sink-input.h
=====================================
@@ -156,6 +156,13 @@ struct pa_sink_input {
* changes. Called from IO context. */
void (*update_max_rewind) (pa_sink_input *i, size_t nbytes); /* may be NULL */
+ /* Called whenever the maximum rewindable size of the sink
+ * changes. Used by virtual sinks to communicate rewind limits
+ * of the virtual sink to the master sink. Must return size_t (-1)
+ * if there is no limit or if the virtual sink is not opened.
+ * Called from IO context. */
+ size_t (*get_max_rewind_limit) (pa_sink_input *i); /* may be NULL */
+
/* Called whenever the maximum request size of the sink
* changes. Called from IO context. */
void (*update_max_request) (pa_sink_input *i, size_t nbytes); /* may be NULL */
@@ -231,6 +238,9 @@ struct pa_sink_input {
* mute status changes. Called from main context */
void (*mute_changed)(pa_sink_input *i); /* may be NULL */
+ /* Used to store the rewind amount of the origin sink during a move */
+ size_t origin_rewind_bytes; /* In sink input sample spec */
+
struct {
pa_sink_input_state_t state;
@@ -252,11 +262,21 @@ struct pa_sink_input {
/* We maintain a history of resampled audio data here. */
pa_memblockq *render_memblockq;
+ /* This queue keeps the history before resampling and is used
+ * when rewinding the resampler. */
+ pa_memblockq *history_memblockq;
+
pa_sink_input *sync_prev, *sync_next;
/* The requested latency for the sink */
pa_usec_t requested_sink_latency;
+ /* Variables used during move */
+ pa_usec_t move_start_time;
+ pa_usec_t origin_sink_latency;
+ size_t resampler_delay_frames;
+ bool dont_rewrite;
+
pa_hashmap *direct_outputs;
} thread_info;
@@ -361,7 +381,7 @@ void pa_sink_input_request_rewind(pa_sink_input *i, size_t nbytes, bool rewrite,
void pa_sink_input_cork(pa_sink_input *i, bool b);
int pa_sink_input_set_rate(pa_sink_input *i, uint32_t rate);
-int pa_sink_input_update_resampler(pa_sink_input *i);
+int pa_sink_input_update_resampler(pa_sink_input *i, bool flush_history);
/* This returns the sink's fields converted into out sample type */
size_t pa_sink_input_get_max_rewind(pa_sink_input *i);
=====================================
src/pulsecore/sink.c
=====================================
@@ -40,6 +40,7 @@
#include <pulsecore/namereg.h>
#include <pulsecore/core-util.h>
#include <pulsecore/sample-util.h>
+#include <pulsecore/stream-util.h>
#include <pulsecore/mix.h>
#include <pulsecore/core-subscribe.h>
#include <pulsecore/log.h>
@@ -338,6 +339,7 @@ pa_sink* pa_sink_new(
s->thread_info.soft_muted = s->muted;
s->thread_info.state = s->state;
s->thread_info.rewind_nbytes = 0;
+ s->thread_info.last_rewind_nbytes = 0;
s->thread_info.rewind_requested = false;
s->thread_info.max_rewind = 0;
s->thread_info.max_request = 0;
@@ -1073,6 +1075,9 @@ void pa_sink_process_rewind(pa_sink *s, size_t nbytes) {
pa_sink_volume_change_rewind(s, nbytes);
}
+ /* Save rewind value */
+ s->thread_info.last_rewind_nbytes = nbytes;
+
PA_HASHMAP_FOREACH(i, s->thread_info.inputs, state) {
pa_sink_input_assert_ref(i);
pa_sink_input_process_rewind(i, nbytes);
@@ -1557,12 +1562,25 @@ void pa_sink_reconfigure(pa_sink *s, pa_sample_spec *spec, bool passthrough) {
PA_IDXSET_FOREACH(i, s->inputs, idx) {
if (i->state == PA_SINK_INPUT_CORKED)
- pa_sink_input_update_resampler(i);
+ pa_sink_input_update_resampler(i, true);
}
pa_sink_suspend(s, false, PA_SUSPEND_INTERNAL);
}
+/* Called from main thread */
+size_t pa_sink_get_last_rewind(pa_sink *s) {
+ size_t rewind_bytes;
+
+ pa_sink_assert_ref(s);
+ pa_assert_ctl_context();
+ pa_assert(PA_SINK_IS_LINKED(s->state));
+
+ pa_assert_se(pa_asyncmsgq_send(s->asyncmsgq, PA_MSGOBJECT(s), PA_SINK_MESSAGE_GET_LAST_REWIND, &rewind_bytes, 0, NULL) == 0);
+
+ return rewind_bytes;
+}
+
/* Called from main thread */
pa_usec_t pa_sink_get_latency(pa_sink *s) {
int64_t usec = 0;
@@ -1639,6 +1657,27 @@ bool pa_sink_flat_volume_enabled(pa_sink *s) {
return false;
}
+/* Check if the sink has a virtual sink attached.
+ * Called from the IO thread. */
+bool pa_sink_has_filter_attached(pa_sink *s) {
+ bool vsink_attached = false;
+ void *state = NULL;
+ pa_sink_input *i;
+
+ pa_assert(s);
+
+ if (PA_SINK_IS_LINKED(s->thread_info.state)) {
+ PA_HASHMAP_FOREACH(i, s->thread_info.inputs, state) {
+ if (!i->origin_sink)
+ continue;
+
+ vsink_attached = true;
+ break;
+ }
+ }
+ return vsink_attached;
+}
+
/* Called from the main thread (and also from the IO thread while the main
* thread is waiting). */
pa_sink *pa_sink_get_master(pa_sink *s) {
@@ -2555,6 +2594,40 @@ static void set_shared_volume_within_thread(pa_sink *s) {
}
}
+/* Called from IO thread. Gets max_rewind limit from sink inputs.
+ * This function is used to communicate the max_rewind value of a
+ * virtual sink to the master sink. The get_max_rewind_limit()
+ * callback is implemented by sink inputs connecting a virtual
+ * sink to its master. */
+static size_t get_max_rewind_limit(pa_sink *s, size_t requested_limit) {
+ pa_sink_input *i;
+ void *state = NULL;
+ size_t rewind_limit;
+
+ pa_assert(s);
+
+ /* Get rewind limit in sink sample spec from sink inputs */
+ rewind_limit = (size_t)(-1);
+ if (PA_SINK_IS_LINKED(s->thread_info.state)) {
+ PA_HASHMAP_FOREACH(i, s->thread_info.inputs, state) {
+
+ if (i->get_max_rewind_limit) {
+ size_t limit;
+
+ limit = i->get_max_rewind_limit(i);
+ if (rewind_limit == (size_t)(-1) || rewind_limit > limit)
+ rewind_limit = limit;
+ }
+ }
+ }
+
+ /* Set max_rewind */
+ if (rewind_limit != (size_t)(-1))
+ requested_limit = PA_MIN(rewind_limit, requested_limit);
+
+ return requested_limit;
+}
+
/* Called from IO thread, except when it is not */
int pa_sink_process_msg(pa_msgobject *o, int code, void *userdata, int64_t offset, pa_memchunk *chunk) {
pa_sink *s = PA_SINK(o);
@@ -2651,8 +2724,8 @@ int pa_sink_process_msg(pa_msgobject *o, int code, void *userdata, int64_t offse
}
pa_hashmap_remove_and_free(s->thread_info.inputs, PA_UINT32_TO_PTR(i->index));
- pa_sink_invalidate_requested_latency(s, true);
pa_sink_request_rewind(s, (size_t) -1);
+ pa_sink_invalidate_requested_latency(s, true);
/* In flat volume mode we need to update the volume as
* well */
@@ -2669,59 +2742,21 @@ int pa_sink_process_msg(pa_msgobject *o, int code, void *userdata, int64_t offse
pa_assert(!i->thread_info.sync_prev);
if (i->thread_info.state != PA_SINK_INPUT_CORKED) {
- pa_usec_t usec = 0;
- size_t sink_nbytes, total_nbytes;
/* The old sink probably has some audio from this
* stream in its buffer. We want to "take it back" as
* much as possible and play it to the new sink. We
* don't know at this point how much the old sink can
- * rewind. We have to pick something, and that
- * something is the full latency of the old sink here.
- * So we rewind the stream buffer by the sink latency
- * amount, which may be more than what we should
- * rewind. This can result in a chunk of audio being
- * played both to the old sink and the new sink.
- *
- * FIXME: Fix this code so that we don't have to make
- * guesses about how much the sink will actually be
- * able to rewind. If someone comes up with a solution
- * for this, something to note is that the part of the
- * latency that the old sink couldn't rewind should
- * ideally be compensated after the stream has moved
- * to the new sink by adding silence. The new sink
- * most likely can't start playing the moved stream
- * immediately, and that gap should be removed from
- * the "compensation silence" (at least at the time of
- * writing this, the move finish code will actually
- * already take care of dropping the new sink's
- * unrewindable latency, so taking into account the
- * unrewindable latency of the old sink is the only
- * problem).
- *
- * The render_memblockq contents are discarded,
- * because when the sink changes, the format of the
- * audio stored in the render_memblockq may change
- * too, making the stored audio invalid. FIXME:
- * However, the read and write indices are moved back
- * the same amount, so if they are not the same now,
- * they won't be the same after the rewind either. If
- * the write index of the render_memblockq is ahead of
- * the read index, then the render_memblockq will feed
- * the new sink some silence first, which it shouldn't
- * do. The write index should be flushed to be the
- * same as the read index. */
-
- /* Get the latency of the sink */
- usec = pa_sink_get_latency_within_thread(s, false);
- sink_nbytes = pa_usec_to_bytes(usec, &s->sample_spec);
- total_nbytes = sink_nbytes + pa_memblockq_get_length(i->thread_info.render_memblockq);
-
- if (total_nbytes > 0) {
- i->thread_info.rewrite_nbytes = i->thread_info.resampler ? pa_resampler_request(i->thread_info.resampler, total_nbytes) : total_nbytes;
- i->thread_info.rewrite_flush = true;
- pa_sink_input_process_rewind(i, sink_nbytes);
- }
+ * rewind, so we just save some values and reconstruct
+ * the render memblockq in finish_move(). */
+
+ /* Save some current values for restore_render_memblockq() */
+ i->thread_info.origin_sink_latency = pa_sink_get_latency_within_thread(s, false);
+ i->thread_info.move_start_time = pa_rtclock_now();
+ i->thread_info.resampler_delay_frames = 0;
+ if (i->thread_info.resampler)
+ /* Round down */
+ i->thread_info.resampler_delay_frames = pa_resampler_get_delay(i->thread_info.resampler, false);
}
pa_sink_input_detach(i);
@@ -2729,11 +2764,13 @@ int pa_sink_process_msg(pa_msgobject *o, int code, void *userdata, int64_t offse
/* Let's remove the sink input ...*/
pa_hashmap_remove_and_free(s->thread_info.inputs, PA_UINT32_TO_PTR(i->index));
- pa_sink_invalidate_requested_latency(s, true);
-
+ /* The rewind must be requested before invalidating the latency, otherwise
+ * the max_rewind value of the sink may change before the rewind. */
pa_log_debug("Requesting rewind due to started move");
pa_sink_request_rewind(s, (size_t) -1);
+ pa_sink_invalidate_requested_latency(s, true);
+
/* In flat volume mode we need to update the volume as
* well */
return o->process_msg(o, PA_SINK_MESSAGE_SET_SHARED_VOLUME, NULL, 0, NULL);
@@ -2754,7 +2791,7 @@ int pa_sink_process_msg(pa_msgobject *o, int code, void *userdata, int64_t offse
if (i->thread_info.state != PA_SINK_INPUT_CORKED) {
pa_usec_t usec = 0;
- size_t nbytes;
+ size_t nbytes, delay_bytes;
/* In the ideal case the new sink would start playing
* the stream immediately. That requires the sink to
@@ -2778,8 +2815,20 @@ int pa_sink_process_msg(pa_msgobject *o, int code, void *userdata, int64_t offse
usec = pa_sink_get_latency_within_thread(s, false);
nbytes = pa_usec_to_bytes(usec, &s->sample_spec);
- if (nbytes > 0)
- pa_sink_input_drop(i, nbytes);
+ /* Calculate number of samples that have been played during the move */
+ delay_bytes = 0;
+ if (i->thread_info.move_start_time > 0) {
+ usec = pa_rtclock_now() - i->thread_info.move_start_time;
+ pa_log_debug("Move took %lu usec", usec);
+ delay_bytes = pa_usec_to_bytes(usec, &s->sample_spec);
+ }
+
+ /* max_rewind must be updated for the sink input because otherwise
+ * the data in the render memblockq will get lost */
+ pa_sink_input_update_max_rewind(i, nbytes);
+
+ if (nbytes + delay_bytes > 0)
+ pa_sink_input_drop(i, nbytes + delay_bytes);
pa_log_debug("Requesting rewind due to finished move");
pa_sink_request_rewind(s, nbytes);
@@ -2796,6 +2845,11 @@ int pa_sink_process_msg(pa_msgobject *o, int code, void *userdata, int64_t offse
pa_sink_input_update_max_rewind(i, s->thread_info.max_rewind);
pa_sink_input_update_max_request(i, s->thread_info.max_request);
+ /* Reset move variables */
+ i->thread_info.move_start_time = 0;
+ i->thread_info.resampler_delay_frames = 0;
+ i->thread_info.origin_sink_latency = 0;
+
return o->process_msg(o, PA_SINK_MESSAGE_SET_SHARED_VOLUME, NULL, 0, NULL);
}
@@ -2942,6 +2996,11 @@ int pa_sink_process_msg(pa_msgobject *o, int code, void *userdata, int64_t offse
*((size_t*) userdata) = s->thread_info.max_rewind;
return 0;
+ case PA_SINK_MESSAGE_GET_LAST_REWIND:
+
+ *((size_t*) userdata) = s->thread_info.last_rewind_nbytes;
+ return 0;
+
case PA_SINK_MESSAGE_GET_MAX_REQUEST:
*((size_t*) userdata) = s->thread_info.max_request;
@@ -3118,6 +3177,8 @@ void pa_sink_set_max_rewind_within_thread(pa_sink *s, size_t max_rewind) {
pa_sink_assert_ref(s);
pa_sink_assert_io_context(s);
+ max_rewind = get_max_rewind_limit(s, max_rewind);
+
if (max_rewind == s->thread_info.max_rewind)
return;
=====================================
src/pulsecore/sink.h
=====================================
@@ -299,6 +299,9 @@ struct pa_sink {
size_t rewind_nbytes;
bool rewind_requested;
+ /* Size of last rewind */
+ size_t last_rewind_nbytes;
+
/* Both dynamic and fixed latencies will be clamped to this
* range. */
pa_usec_t min_latency; /* we won't go below this latency */
@@ -359,6 +362,7 @@ typedef enum pa_sink_message {
PA_SINK_MESSAGE_SET_MAX_REQUEST,
PA_SINK_MESSAGE_UPDATE_VOLUME_AND_MUTE,
PA_SINK_MESSAGE_SET_PORT_LATENCY_OFFSET,
+ PA_SINK_MESSAGE_GET_LAST_REWIND,
PA_SINK_MESSAGE_MAX
} pa_sink_message_t;
@@ -456,6 +460,7 @@ void pa_sink_get_latency_range(pa_sink *s, pa_usec_t *min_latency, pa_usec_t *ma
pa_usec_t pa_sink_get_fixed_latency(pa_sink *s);
size_t pa_sink_get_max_rewind(pa_sink *s);
+size_t pa_sink_get_last_rewind(pa_sink *s);
size_t pa_sink_get_max_request(pa_sink *s);
int pa_sink_update_status(pa_sink*s);
@@ -465,6 +470,10 @@ int pa_sink_suspend_all(pa_core *c, bool suspend, pa_suspend_cause_t cause);
/* Use this instead of checking s->flags & PA_SINK_FLAT_VOLUME directly. */
bool pa_sink_flat_volume_enabled(pa_sink *s);
+/* Check if the sink has a virtual sink attached.
+ * Called from the IO thread. */
+bool pa_sink_has_filter_attached(pa_sink *s);
+
/* Get the master sink when sharing volumes */
pa_sink *pa_sink_get_master(pa_sink *s);
=====================================
src/pulsecore/source-output.c
=====================================
@@ -29,6 +29,7 @@
#include <pulse/xmalloc.h>
#include <pulse/util.h>
#include <pulse/internal.h>
+#include <pulse/timeval.h>
#include <pulsecore/core-format.h>
#include <pulsecore/mix.h>
@@ -237,6 +238,7 @@ int pa_source_output_new(
pa_channel_map volume_map;
int r;
char *pt;
+ size_t resampler_history;
pa_assert(_o);
pa_assert(core);
@@ -499,6 +501,11 @@ int pa_source_output_new(
0,
&o->source->silence);
+ resampler_history = (uint64_t) PA_RESAMPLER_MAX_DELAY_USEC * o->source->sample_spec.rate / PA_USEC_PER_SEC;
+ resampler_history *= pa_frame_size(&o->source->sample_spec);
+
+ pa_memblockq_set_maxrewind(o->thread_info.delay_memblockq, resampler_history + pa_source_get_max_rewind(o->source));
+
pa_assert_se(pa_idxset_put(core->source_outputs, o, &o->index) == 0);
pa_assert_se(pa_idxset_put(o->source->outputs, pa_source_output_ref(o), NULL) == 0);
@@ -865,21 +872,36 @@ void pa_source_output_process_rewind(pa_source_output *o, size_t nbytes /* in so
return;
if (o->process_rewind) {
- pa_assert(pa_memblockq_get_length(o->thread_info.delay_memblockq) == 0);
+ size_t source_output_nbytes;
+ size_t length;
- if (o->thread_info.resampler)
- nbytes = pa_resampler_result(o->thread_info.resampler, nbytes);
+ /* The length of the memblockq may be non-zero if pa_source_output_rewind() is called twice
+ * without pa_source_output_push() called in between. In that case, the resampler has already
+ * been reset and we can skip that part. */
+ length = pa_memblockq_get_length(o->thread_info.delay_memblockq);
- pa_log_debug("Have to rewind %lu bytes on implementor.", (unsigned long) nbytes);
+ pa_memblockq_rewind(o->thread_info.delay_memblockq, nbytes);
- if (nbytes > 0)
- o->process_rewind(o, nbytes);
+ source_output_nbytes = pa_resampler_result(o->thread_info.resampler, nbytes);
- if (o->thread_info.resampler)
- pa_resampler_rewind(o->thread_info.resampler, nbytes);
+ pa_log_debug("Have to rewind %lu bytes on implementor.", (unsigned long) source_output_nbytes);
- } else
- pa_memblockq_seek(o->thread_info.delay_memblockq, - ((int64_t) nbytes), PA_SEEK_RELATIVE, true);
+ if (source_output_nbytes > 0)
+ o->process_rewind(o, source_output_nbytes);
+
+ if (o->thread_info.resampler && length == 0) {
+ size_t resampler_bytes;
+
+ /* Round down to full frames */
+ resampler_bytes = (size_t) pa_resampler_get_delay(o->thread_info.resampler, false) * pa_frame_size(&o->source->sample_spec);
+ if (resampler_bytes > 0)
+ pa_memblockq_rewind(o->thread_info.delay_memblockq, resampler_bytes);
+
+ pa_resampler_rewind(o->thread_info.resampler, source_output_nbytes, NULL, 0);
+ }
+ }
+
+ pa_memblockq_seek(o->thread_info.delay_memblockq, - ((int64_t) nbytes), PA_SEEK_RELATIVE, true);
}
/* Called from thread context */
@@ -887,18 +909,25 @@ size_t pa_source_output_get_max_rewind(pa_source_output *o) {
pa_source_output_assert_ref(o);
pa_source_output_assert_io_context(o);
- return o->thread_info.resampler ? pa_resampler_request(o->thread_info.resampler, o->source->thread_info.max_rewind) : o->source->thread_info.max_rewind;
+ return pa_resampler_result(o->thread_info.resampler, o->source->thread_info.max_rewind);
}
/* Called from thread context */
void pa_source_output_update_max_rewind(pa_source_output *o, size_t nbytes /* in the source's sample spec */) {
+ size_t resampler_history;
+
pa_source_output_assert_ref(o);
pa_source_output_assert_io_context(o);
pa_assert(PA_SOURCE_OUTPUT_IS_LINKED(o->thread_info.state));
pa_assert(pa_frame_aligned(nbytes, &o->source->sample_spec));
+ resampler_history = (uint64_t) PA_RESAMPLER_MAX_DELAY_USEC * o->source->sample_spec.rate / PA_USEC_PER_SEC;
+ resampler_history *= pa_frame_size(&o->source->sample_spec);
+
+ pa_memblockq_set_maxrewind(o->thread_info.delay_memblockq, resampler_history + nbytes);
+
if (o->update_max_rewind)
- o->update_max_rewind(o, o->thread_info.resampler ? pa_resampler_result(o->thread_info.resampler, nbytes) : nbytes);
+ o->update_max_rewind(o, pa_resampler_result(o->thread_info.resampler, nbytes));
}
/* Called from thread context */
@@ -1701,6 +1730,7 @@ int pa_source_output_process_msg(pa_msgobject *mo, int code, void *userdata, int
pa_usec_t *r = userdata;
r[0] += pa_bytes_to_usec(pa_memblockq_get_length(o->thread_info.delay_memblockq), &o->source->sample_spec);
+ r[0] += pa_resampler_get_delay_usec(o->thread_info.resampler);
r[1] += pa_source_get_latency_within_thread(o->source, false);
return 0;
@@ -1784,6 +1814,7 @@ finish:
int pa_source_output_update_resampler(pa_source_output *o) {
pa_resampler *new_resampler;
char *memblockq_name;
+ size_t resampler_history;
pa_source_output_assert_ref(o);
pa_assert_ctl_context();
@@ -1841,6 +1872,11 @@ int pa_source_output_update_resampler(pa_source_output *o) {
&o->source->silence);
pa_xfree(memblockq_name);
+ resampler_history = (uint64_t) PA_RESAMPLER_MAX_DELAY_USEC * o->source->sample_spec.rate / PA_USEC_PER_SEC;
+ resampler_history *= pa_frame_size(&o->source->sample_spec);
+
+ pa_memblockq_set_maxrewind(o->thread_info.delay_memblockq, resampler_history + pa_source_get_max_rewind(o->source));
+
o->actual_resample_method = new_resampler ? pa_resampler_get_method(new_resampler) : PA_RESAMPLER_INVALID;
pa_log_debug("Updated resampler for source output %d", o->index);
=====================================
src/tests/meson.build
=====================================
@@ -70,6 +70,8 @@ if get_option('daemon')
[ check_dep, libpulse_dep, libpulsecommon_dep, libpulsecore_dep ] ],
[ 'resampler-test', 'resampler-test.c',
[ libpulse_dep, libpulsecommon_dep, libpulsecore_dep, libintl_dep ] ],
+ [ 'resampler-rewind-test', 'resampler-rewind-test.c',
+ [ libpulse_dep, libpulsecommon_dep, libpulsecore_dep, libintl_dep, libm_dep ] ],
[ 'rtpoll-test', 'rtpoll-test.c',
[ check_dep, libpulse_dep, libpulsecommon_dep, libpulsecore_dep ] ],
[ 'smoother-test', 'smoother-test.c',
=====================================
src/tests/resampler-rewind-test.c
=====================================
@@ -0,0 +1,437 @@
+/***
+ This file is part of PulseAudio.
+
+ PulseAudio is free software; you can redistribute it and/or modify
+ it under the terms of the GNU Lesser General Public License as published
+ by the Free Software Foundation; either version 2.1 of the License,
+ or (at your option) any later version.
+
+ PulseAudio is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public License
+ along with PulseAudio; if not, see <http://www.gnu.org/licenses/>.
+***/
+
+#ifdef HAVE_CONFIG_H
+#include <config.h>
+#endif
+
+#include <stdio.h>
+#include <getopt.h>
+#include <locale.h>
+#include <math.h>
+
+#include <pulse/pulseaudio.h>
+
+#include <pulse/rtclock.h>
+#include <pulse/sample.h>
+#include <pulse/volume.h>
+
+#include <pulsecore/i18n.h>
+#include <pulsecore/log.h>
+#include <pulsecore/resampler.h>
+#include <pulsecore/macro.h>
+#include <pulsecore/endianmacros.h>
+#include <pulsecore/memblock.h>
+#include <pulsecore/memblockq.h>
+#include <pulsecore/sample-util.h>
+#include <pulsecore/core-util.h>
+
+#define MEMBLOCKQ_MAXLENGTH (16*1024*1024)
+#define PA_SILENCE_MAX (pa_page_size()*16)
+#define MAX_MATCHING_PERIOD 500
+
+static pa_memblock *silence_memblock_new(pa_mempool *pool, uint8_t c) {
+ pa_memblock *b;
+ size_t length;
+ void *data;
+
+ pa_assert(pool);
+
+ length = PA_MIN(pa_mempool_block_size_max(pool), PA_SILENCE_MAX);
+
+ b = pa_memblock_new(pool, length);
+
+ data = pa_memblock_acquire(b);
+ memset(data, c, length);
+ pa_memblock_release(b);
+
+ pa_memblock_set_is_silence(b, true);
+
+ return b;
+}
+
+/* Calculate number of history bytes needed for the rewind */
+static size_t calculate_resampler_history_bytes(pa_resampler *r, size_t in_rewind_frames) {
+ size_t history_frames, history_max, matching_period, total_frames, remainder;
+ double delay;
+
+ if (!r)
+ return 0;
+
+ /* Initialize some variables, cut off full seconds from the rewind */
+ total_frames = 0;
+ in_rewind_frames = in_rewind_frames % r->i_ss.rate;
+ history_max = (uint64_t) PA_RESAMPLER_MAX_DELAY_USEC * r->i_ss.rate * 3 / PA_USEC_PER_SEC / 2;
+
+ /* Get the current internal delay of the resampler */
+ delay = pa_resampler_get_delay(r, false);
+
+ /* Calculate the matchiung period */
+ matching_period = r->i_ss.rate / pa_resampler_get_gcd(r);
+ pa_log_debug("Integral period length is %lu input frames", matching_period);
+
+ /* If the delay is larger than the length of the history queue, we can only
+ * replay as much as we have. */
+ if ((size_t)delay >= history_max) {
+ history_frames = history_max;
+ return history_frames * r->i_fz;
+ }
+
+ /* Initially set the history to 3 times the resampler delay. Use at least 2 ms. */
+ history_frames = (size_t)(delay * 3.0);
+ history_frames = PA_MAX(history_frames, r->i_ss.rate / 500);
+
+ /* Check how the rewind fits into multiples of the matching period. */
+ remainder = (in_rewind_frames + history_frames) % matching_period;
+
+ /* If possible, use between 2 and 3 times the resampler delay */
+ if (remainder < (size_t)delay && history_frames - remainder <= history_max)
+ total_frames = in_rewind_frames + history_frames - remainder;
+
+ /* Else, try above 3 times the delay */
+ else if (history_frames + matching_period - remainder <= history_max)
+ total_frames = in_rewind_frames + history_frames + matching_period - remainder;
+
+ if (total_frames != 0)
+ /* We found a perfect match. */
+ history_frames = total_frames - in_rewind_frames;
+ else {
+ /* Try to use 2.5 times the delay. */
+ history_frames = PA_MIN((size_t)(delay * 2.5), history_max);
+ pa_log_debug("No usable integral matching period");
+ }
+
+ return history_frames * r->i_fz;
+}
+
+static float compare_blocks(const pa_sample_spec *ss, const pa_memchunk *chunk_a, const pa_memchunk *chunk_b) {
+ float *a, *b, max_diff = 0;
+ unsigned i;
+
+ a = pa_memblock_acquire(chunk_a->memblock);
+ b = pa_memblock_acquire(chunk_b->memblock);
+ a += chunk_a->index / pa_frame_size(ss);
+ b += chunk_b->index / pa_frame_size(ss);
+
+ for (i = 0; i < chunk_a->length / pa_frame_size(ss); i++) {
+ if (fabs(a[i] - b[i]) > max_diff)
+ max_diff = fabs(a[i] - b[i]);
+ }
+
+ pa_memblock_release(chunk_a->memblock);
+ pa_memblock_release(chunk_b->memblock);
+
+ return max_diff;
+}
+
+static pa_memblock* generate_block(pa_mempool *pool, const pa_sample_spec *ss, unsigned frequency, double amplitude, size_t nr_of_samples) {
+ pa_memblock *r;
+ float *d;
+ float val;
+ unsigned i;
+ int n;
+ float t, dt, dt_period;
+
+ pa_assert(frequency);
+ pa_assert(nr_of_samples);
+ pa_assert(ss->channels == 1);
+ pa_assert(ss->format == PA_SAMPLE_FLOAT32NE);
+
+ pa_assert_se(r = pa_memblock_new(pool, pa_frame_size(ss) * nr_of_samples));
+ d = pa_memblock_acquire(r);
+
+ /* Generate square wave with given length, frequency and sample rate. */
+ val = amplitude;
+ t = 0;
+ n = 1;
+ dt = 1.0 / ss->rate;
+ dt_period = 1.0 / frequency;
+ for (i=0; i < nr_of_samples; i++) {
+ d[i] = val;
+
+ if ((int)(2 * t / dt_period) >= n) {
+ n++;
+ if (val >= amplitude)
+ val = - amplitude;
+ else
+ val = amplitude;
+ }
+
+ t += dt;
+ }
+
+ pa_memblock_release(r);
+
+ return r;
+}
+
+static void help(const char *argv0) {
+ printf("%s [options]\n\n"
+ "-h, --help Show this help\n"
+ "-v, --verbose Print debug messages\n"
+ " --from-rate=SAMPLERATE From sample rate in Hz (defaults to 44100)\n"
+ " --to-rate=SAMPLERATE To sample rate in Hz (defaults to 44100)\n"
+ " --resample-method=METHOD Resample method (defaults to auto)\n"
+ " --frequency=unsigned Frequency of square wave\n"
+ " --samples=unsigned Number of samples for square wave\n"
+ " --rewind=unsigned Number of output samples to rewind\n"
+ "\n"
+ "This test generates samples for a square wave of given frequency, number of samples\n"
+ "and input sample rate. Then this input data is resampled to the output rate, rewound\n"
+ "by rewind samples and the rewound part is processed again. Then output is compared to\n"
+ "the result of the first pass.\n"
+ "\n"
+ "See --dump-resample-methods for possible values of resample methods.\n",
+ argv0);
+}
+
+enum {
+ ARG_VERSION = 256,
+ ARG_FROM_SAMPLERATE,
+ ARG_TO_SAMPLERATE,
+ ARG_FREQUENCY,
+ ARG_SAMPLES,
+ ARG_REWIND,
+ ARG_RESAMPLE_METHOD,
+ ARG_DUMP_RESAMPLE_METHODS
+};
+
+static void dump_resample_methods(void) {
+ int i;
+
+ for (i = 0; i < PA_RESAMPLER_MAX; i++)
+ if (pa_resample_method_supported(i))
+ printf("%s\n", pa_resample_method_to_string(i));
+
+}
+
+int main(int argc, char *argv[]) {
+ pa_mempool *pool = NULL;
+ pa_sample_spec a, b;
+ pa_resample_method_t method;
+ int ret = 1, c;
+ unsigned samples, frequency, rewind;
+ unsigned crossover_freq = 120;
+ pa_resampler *resampler;
+ pa_memchunk in_chunk, out_chunk, rewound_chunk, silence_chunk;
+ pa_usec_t ts;
+ pa_memblockq *history_queue = NULL;
+ size_t in_rewind_size, in_frame_size, history_size, out_rewind_size, old_length, in_resampler_buffer, n_out_expected;
+ float max_diff;
+ double delay_before, delay_after, delay_expected;
+
+ static const struct option long_options[] = {
+ {"help", 0, NULL, 'h'},
+ {"verbose", 0, NULL, 'v'},
+ {"version", 0, NULL, ARG_VERSION},
+ {"from-rate", 1, NULL, ARG_FROM_SAMPLERATE},
+ {"to-rate", 1, NULL, ARG_TO_SAMPLERATE},
+ {"frequency", 1, NULL, ARG_FREQUENCY},
+ {"samples", 1, NULL, ARG_SAMPLES},
+ {"rewind", 1, NULL, ARG_REWIND},
+ {"resample-method", 1, NULL, ARG_RESAMPLE_METHOD},
+ {"dump-resample-methods", 0, NULL, ARG_DUMP_RESAMPLE_METHODS},
+ {NULL, 0, NULL, 0}
+ };
+
+ setlocale(LC_ALL, "");
+#ifdef ENABLE_NLS
+ bindtextdomain(GETTEXT_PACKAGE, PULSE_LOCALEDIR);
+#endif
+
+ pa_log_set_level(PA_LOG_WARN);
+ if (!getenv("MAKE_CHECK"))
+ pa_log_set_level(PA_LOG_INFO);
+
+ a.channels = b.channels = 1;
+ a.rate = 48000;
+ b.rate = 44100;
+ a.format = b.format = PA_SAMPLE_FLOAT32NE;
+
+ method = PA_RESAMPLER_AUTO;
+ frequency = 1000;
+ samples = 5000;
+ rewind = 2500;
+
+ while ((c = getopt_long(argc, argv, "hv", long_options, NULL)) != -1) {
+
+ switch (c) {
+ case 'h' :
+ help(argv[0]);
+ ret = 0;
+ goto quit;
+
+ case 'v':
+ pa_log_set_level(PA_LOG_DEBUG);
+ break;
+
+ case ARG_VERSION:
+ printf("%s %s\n", argv[0], PACKAGE_VERSION);
+ ret = 0;
+ goto quit;
+
+ case ARG_DUMP_RESAMPLE_METHODS:
+ dump_resample_methods();
+ ret = 0;
+ goto quit;
+
+ case ARG_FROM_SAMPLERATE:
+ a.rate = (uint32_t) atoi(optarg);
+ break;
+
+ case ARG_TO_SAMPLERATE:
+ b.rate = (uint32_t) atoi(optarg);
+ break;
+
+ case ARG_FREQUENCY:
+ frequency = (unsigned) atoi(optarg);
+ break;
+
+ case ARG_SAMPLES:
+ samples = (unsigned) atoi(optarg);
+ break;
+
+ case ARG_REWIND:
+ rewind = (unsigned) atoi(optarg);
+ break;
+
+ case ARG_RESAMPLE_METHOD:
+ if (*optarg == '\0' || pa_streq(optarg, "help")) {
+ dump_resample_methods();
+ ret = 0;
+ goto quit;
+ }
+ method = pa_parse_resample_method(optarg);
+ break;
+
+ default:
+ goto quit;
+ }
+ }
+
+ pa_log_info("=== Square wave %u Hz, %u samples. Resampling using %s from %u Hz to %u Hz, rewinding %u output samples.", frequency,
+ samples, pa_resample_method_to_string(method), a.rate, b.rate, rewind);
+
+ ret = 0;
+ pa_assert_se(pool = pa_mempool_new(PA_MEM_TYPE_PRIVATE, 0, true));
+
+ pa_log_debug("Compilation CFLAGS: %s", PA_CFLAGS);
+
+ /* Setup resampler */
+ ts = pa_rtclock_now();
+ pa_assert_se(resampler = pa_resampler_new(pool, &a, NULL, &b, NULL, crossover_freq, method, 0));
+ pa_log_info("Init took %llu usec", (long long unsigned)(pa_rtclock_now() - ts));
+
+ /* Generate input data */
+ in_chunk.memblock = generate_block(pool, &a, frequency, 0.5, samples);
+ in_chunk.length = pa_memblock_get_length(in_chunk.memblock);
+ in_chunk.index = 0;
+ in_frame_size = pa_frame_size(&a);
+
+ /* First, resample the full block */
+ ts = pa_rtclock_now();
+ pa_resampler_run(resampler, &in_chunk, &out_chunk);
+ if (!out_chunk.memblock) {
+ pa_memblock_unref(in_chunk.memblock);
+ pa_log_warn("Resampling did not return any output data");
+ ret = 1;
+ goto quit;
+ }
+
+ pa_log_info("resampling took %llu usec.", (long long unsigned)(pa_rtclock_now() - ts));
+ if (rewind > out_chunk.length / pa_frame_size(&b)) {
+ pa_log_warn("Specified number of frames to rewind (%u) larger than number of output frames (%lu), aborting.", rewind, out_chunk.length / pa_frame_size(&b));
+ ret = 1;
+ goto quit1;
+ }
+
+ /* Get delay after first resampling pass */
+ delay_before = pa_resampler_get_delay(resampler, true);
+
+ /* Create and prepare history queue */
+ silence_chunk.memblock = silence_memblock_new(pool, 0);
+ silence_chunk.length = pa_frame_align(pa_memblock_get_length(silence_chunk.memblock), &a);
+ silence_chunk.index = 0;
+ history_queue = pa_memblockq_new("Test-Queue", 0, MEMBLOCKQ_MAXLENGTH, 0, &a, 0, 1, samples * in_frame_size, &silence_chunk);
+ pa_memblock_unref(silence_chunk.memblock);
+
+ pa_memblockq_push(history_queue, &in_chunk);
+ pa_memblockq_drop(history_queue, samples * in_frame_size);
+
+ in_rewind_size = pa_resampler_request(resampler, rewind * pa_frame_size(&b));
+ out_rewind_size = rewind * pa_frame_size(&b);
+ pa_log_debug("Have to rewind %lu input frames", in_rewind_size / in_frame_size);
+ ts = pa_rtclock_now();
+
+ /* Now rewind the resampler */
+ pa_memblockq_rewind(history_queue, in_rewind_size);
+ history_size = calculate_resampler_history_bytes(resampler, in_rewind_size / in_frame_size);
+ pa_log_debug("History is %lu frames.", history_size / in_frame_size);
+ pa_resampler_rewind(resampler, out_rewind_size, history_queue, history_size);
+
+ pa_log_info("Rewind took %llu usec.", (long long unsigned)(pa_rtclock_now() - ts));
+ ts = pa_rtclock_now();
+
+ /* Re-run the resampler */
+ old_length = in_chunk.length;
+ in_chunk.length = in_rewind_size;
+ in_chunk.index = old_length - in_chunk.length;
+ pa_resampler_run(resampler, &in_chunk, &rewound_chunk);
+ if (!rewound_chunk.memblock) {
+ pa_log_warn("Resampler did not return output data for rewind");
+ ret = 1;
+ goto quit1;
+ }
+
+ /* Get delay after rewind */
+ delay_after = pa_resampler_get_delay(resampler, true);
+
+ /* Calculate expected delay */
+ n_out_expected = pa_resampler_result(resampler, in_rewind_size + history_size) / pa_frame_size(&b);
+ delay_expected = delay_before + (double)(in_rewind_size + history_size) / (double)in_frame_size - n_out_expected * (double)a.rate / (double)b.rate;
+
+ /* Check for leftover samples in the resampler buffer */
+ in_resampler_buffer = lround((delay_after - delay_expected) * (double)b.rate / (double)a.rate);
+ if (in_resampler_buffer != 0) {
+ pa_log_debug("%li output frames still in resampler buffer", in_resampler_buffer);
+ }
+
+ pa_log_info("Second resampler run took %llu usec.", (long long unsigned)(pa_rtclock_now() - ts));
+ pa_log_debug("Got %lu output frames", rewound_chunk.length / pa_frame_size(&b));
+ old_length = out_chunk.length;
+ out_chunk.length = rewound_chunk.length;
+ out_chunk.index = old_length - out_chunk.length;
+
+ max_diff = compare_blocks(&b, &out_chunk, &rewound_chunk);
+ pa_log_info("Maximum difference is %.*g", 6, max_diff);
+
+ pa_memblock_unref(rewound_chunk.memblock);
+
+quit1:
+ pa_memblock_unref(in_chunk.memblock);
+ pa_memblock_unref(out_chunk.memblock);
+
+ pa_resampler_free(resampler);
+ if (history_queue)
+ pa_memblockq_free(history_queue);
+
+quit:
+ if (pool)
+ pa_mempool_unref(pool);
+
+ return ret;
+}
View it on GitLab: https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/compare/90ccfc1688cdf0b80c168702c43340581674ae54...851c377d6b4cbcf4cadd142445f87207d080c770
--
View it on GitLab: https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/compare/90ccfc1688cdf0b80c168702c43340581674ae54...851c377d6b4cbcf4cadd142445f87207d080c770
You're receiving this email because of your account on gitlab.freedesktop.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/pulseaudio-commits/attachments/20211103/65d7a494/attachment-0001.htm>
More information about the pulseaudio-commits
mailing list