[pulseaudio-discuss] LFE remixing

[Tanu Kaskinen]

Hi,

One of the proposed topics for the miniconf is how to separate the low 
frequencies for the LFE channel when remixing. When discussing GSoC 
projects, one of Alexander's proposals was to implement advanced 
remixing features (beyond just LFE remixing), and at that point I made 
this comment[1]:

"The first thing to do is to add the infrastructure
for supporting multiple remixer implementations. Then a hook should be
added to make it possible to choose the remixer implementation on a
case-by-case basis from a policy module. Then we can start adding fancy
remixer implementations."

Alexander asked me to elaborate on that comment, as a preparation for 
the LFE remixing discussion, so here we go:

First of all, I don't think this new infrastructure is necessary if the 
immediate goal is just to add low-pass filtering when synthesizing a LFE 
channel in the current remixer. If there exists only one remixer 
implementation and the policy for choosing the parameters is fixed 
(controllable with global options in daemon.conf), there are no modules 
that would use this infrastructure.

We will hopefully support multiple remixer implementations in the 
future, however. For example, I believe the best way to integrate a 
virtual surround filter would be via a new remixer implementation. Also, 
if we want to set different remixing parameters for different devices, 
that will require a new interface for setting the remixer parameters of 
a stream even if we only have a single remixer implementation.

So, there's need for different implementations of remixer objects, and 
the implementations require parameters. The parameters are likely very 
different between remixer implementations. When we need to create a 
remixer, we need to choose the algorithm and the parameters. This 
information can be passed to pa_resampler_new() via two string 
parameters: the remixer implementation identifier and the parameter 
preset name. This can be later generalized so that remixers are just one 
class of filters, and the parameter preset handling can be unified for 
all filters.

(A sidenote: I think remixing and sample format conversion should be 
eventually taken out of pa_resampler into separate filter objects, but I 
don't think that's necessary at this point.)

When pa_resampler_new() gets the remixer implementation identifier and 
the preset name, it uses the implementation identifier to get a factory 
object from some registry, and then uses the factory to instantiate the 
remixer, passing the preset to set the initial remixer parameters.

We then need some policy for choosing the remixer implementation and the 
preset when a new stream is created, or a stream is moved to a different 
device. It might make sense at some point to add a hook that policy 
modules could use to do the choice, but we could start with a fixed 
policy: if the device has the remixer id and preset name set, then use 
those for the resampler, otherwise use the default remixer as configured 
in daemon.conf. Module-card-restore can be extended so that it's capable 
of saving and restoring the remixer id and preset name for each port.

The actual contents of the presets (i.e. the individual parameter 
values) could be managed by module-filter-preset-restore, but I'd 
suggest doing that in the core instead (and using ini-style files for 
storage instead of pa_database). I don't really see the value in using 
the "restore module pattern" for implementing persistence.

A summary of my proposal:

To add LFE low-pass filtering, just hack the current remixer 
implementation, no need for interface changes anywhere.

To make the remixer parameters configurable per device:
  - Add two string attributes to ports, sinks and sources: the remixer 
implementation id and the preset name.
  - Modify module-card-restore and module-device-restore so that they 
save and restore the new attributes.
  - Implement a preset database.
  - Modify pa_resampler so that its remixing behaviour can be controlled 
by giving the remixer implementation id and the preset name to 
pa_resampler_new.
  - Modify pa_sink_input and pa_source_output so that they use the 
device's remixer attributes when calling pa_resampler_new().

To support multiple remixer implementations:
  - Add a registry for remixer implementations so that modules can add 
and remove implementations dynamically.
  - Modify pa_resampler_new() so that it uses the remixer registry when 
initializing the remixer.

Hmm... Now I see one weak point in this proposal: adding the new 
attributes to ports, sinks and sources is perhaps not a very good idea, 
because we might actually end up with a policy that uses some other 
information for choosing the remixer than just the device, in which case 
the new attributes would be meaningless or at least insufficient. For 
example, whether or not to enable the virtual surround remixer depends 
also on the stream channel map.

Perhaps it would be better to not add those attributes to ports, sinks 
and sources, but instead add module-remixing-policy, which could 
implement several policy models. The first implemented model would only 
use the device to choose the remixer. Saving and restoring the remixer 
attributes for each device would be done in module-remixing-policy 
instead of module-card/device-restore.

-- 
Tanu