[pulseaudio-discuss] R: New equalizer module (module-eqpro-sink), some questions

Fri Apr 19 14:56:11 UTC 2019

On Fri, 2019-04-19 at 12:03 +0200, Georg Chini wrote:
> On 19.04.19 11:13, Tanu Kaskinen wrote:
> > On Tue, 2019-04-16 at 21:40 +0200, Georg Chini wrote:
> > > On 16.04.19 19:19, Tanu Kaskinen wrote:
> > > > On Thu, 2019-04-11 at 20:42 +0200, Georg Chini wrote:
> > > > > On 11.04.19 19:36, Tanu Kaskinen wrote:
> > > > > > If you want a better plugin standard, are you aware of LV2
> > > > > > and PipeWire's SPA (the latter doesn't seem to be properly documented
> > > > > > yet, but to my understanding it's supposed to have a stable and
> > > > > > flexible API)?
> > > > > Arun already suggested the pipewire SPA. I took a look, but it
> > > > > seems not very simple compared to LADSPA. I could not really
> > > > > understand how it works and it appears to support a lot more
> > > > > than just filters.
> > > > LV2 would also be an option, although it too is pretty complex compared
> > > > to LADSPA. But at least it's documented and has examples.
> > > I just took a look and on the first glance LV2 seems similar
> > > to LADSPA. I have to dig into the details though, maybe control
> > > arrays and interleaved audio ports are possible there.
> > I'm pretty sure they are possible, but neither of those features are
> > necessary. If the plugin gets the number of bands during the
> > initialization, it can create the appropriate number of non-array
> > control ports. Interleaved audio ports aren't needed either, because
> > PulseAudio can do the deinterleaving before passing the audio to the
> > plugin (like module-ladspa-sink already does). If one's going to write
> > an LV2 plugin, it's best to use standard port types so that all hosts
> > will be able to use the plugin.
> 
> The problem here is that the number of ports must be known before
> the initialization because it is something which is already provided by
> the plugin descriptor. So there seems to be no way to change that
> number dynamically unless I misread the documentation. But looking
> at the LV2 standard, it supplies the port type lv2:CVPort (see
> https://gitlab.com/lv2/lv2/blob/master/lv2/core/lv2core.ttl, line 256)
> which is what I have been looking for if I read the documentation
> right.

I don't think CVPorts are relevant for this discussion. As far as I can
tell, they just provide a different kind of control port, one that can
use audio signal as input. You wanted a port that can take an array of
values, CVPorts isn't that.

You seem to be right about the requirement to declare all ports in
advance. I thought dynamic ports would be the primary benefit of using
LV2 rather than creating a custom API based on LADSPA.

> Concerning interleaved audio format: Up to now I found nothing
> that explicitly forbids interleaved audio though it appears that the
> plugins usually provide one audio port per channel.

You can't use a plain AudioPort for interleaved audio, because hosts
will assume that it operates with mono audio. You can probably define a
subclass of AudioPort with different semantics, but then hosts other
than PulseAudio won't be able to use the plugin (unless they adopt your
extension).

> PA can surely deinterleave the input and interleave the output
> but to me it looks like completely unnecessary copying around
> of buffers within a real time thread. With interleaved channels,
> you could directly pass the input and output buffers. Why should
> we copy the data twice if it can be avoided? Additionally, using
> one port per channel makes it impossible to adapt the number
> of channels dynamically when loading the plugin for the reason
> given above.

The reason I suggested deinterleaving in PulseAudio was to allow the
plugin to be compatible with other hosts. Without native support for
dynamic ports in LV2, such compatibility seems to be hard/impossible to
achieve, however.

As for dynamically changing channels, I don't see the use case for
that.

> > > > > > You say that your extension allows full integration of Andrea's
> > > > > > equalizer, but I don't see how it allows the host to tell the plugin
> > > > > > how many channels and how many frequency bands it should initialize.
> > > > > For an interleaved audio port, there would be another control
> > > > > port which holds the number of (interleaved) channels. So
> > > > > this port would allow you to change the number of channels.
> > > > > You could for example have an audio port named "Input"
> > > > > and a control port "Number of input channels". Then the
> > > > > get_info_port() function would return the index of the
> > > > > "Number of input channels" control when called with the
> > > > > "Input" port as argument. Or the other way round: If you
> > > > > set "Number of input channels"  to 6 the plugin will expect
> > > > > 6 channels in the interleaved audio port (and you know
> > > > > which control port sets the number of channels because
> > > > > you can get it via the get_info_port() function.
> > > > > 
> > > > > The same applies to the number of bands. There must be a
> > > > > control port which reflects the number of elements in the
> > > > > control array which is the same as the number of bands.
> > > > > 
> > > > > Both values can be set to convenient defaults if the host does
> > > > > not supply them (like 5 bands and 2 channels).
> > > > Ok, so the idea is to do the configuration while the filter is running.
> > > > I think it would be better to do the configuration in the plugin setup
> > > > phase. I imagine that would simplify the control port allocoation and
> > > > management, since the setup doesn't have to run in the IO thread where
> > > > malloc() is not allowed. I don't see much benefit in doing this kind of
> > > > configuration while the filter is running, since the filter state most
> > > > likely has to be reset anyway when the number of EQ bands is changed.
> > > > 
> > > > There could be a function for getting a description of what options the
> > > > plugin accepts, and a setup function for setting the options.
> > > > 
> > > Why do you think that the filter must be configured while it is
> > > running? In case of the equalizer the number of channels and
> > > also the number of bands are known before the filter is run.
> > > The LADSPA standard says all control ports must be connected
> > > (and valid) before you can run the filter. If a parameter changes
> > > at runtime, the filter must be reset like the current ladspa-sink
> > > does.
> > Control ports are used for realtime parameter changes, so that's why I
> > thought the intention was to set the parameters while the filter is
> > running. I think it would be much clearer and easier to document the
> > expected host behaviour if the parameter configuration was not
> > implemented via control ports.
> > 
> Control ports are used for both - initial configuration and changes
> during run time. And yes, the intention is to change parameters
> while the filter is running. As explained in the follow-up mail to
> this one, changes at run-time must be done from the IO thread
> while the initial configuration can be done from the main thread.
> If a parameter changes during run-time, the filter must be reset.
> So you have to distinguish between the two cases of preparing
> the filter for running the first time and changing parameters
> while it is running.

Changing the number of eq bands isn't quite like changing regular
control values. The plugin probably has some per-band data, which has
to be reallocated when the number of bands changes. malloc() isn't
allowed in the IO thread. Also, all gain values assigned to the bands
previously are likely useless, because the band frequencies change. The
host will likely have to set all bands to the default gain value, so in
effect changing the band count is like starting from scratch, which is
very different from changing a regular control value.

If the eq band count was an initialization parameter rather than a
control port, the IO thread limitations wouldn't become an issue, and
it would be explicit that changing the eq band count means starting
from scratch. It should still be possible to change initialization
parameters at runtime, that would just mean that a new plugin instance
is created and the old instance is removed.

-- 
Tanu

https://www.patreon.com/tanuk
https://liberapay.com/tanuk