[pulseaudio-discuss] RFC: New volume functionality for PulseAudio

[David Henningsson]

On 02/14/2014 10:49 AM, Tanu Kaskinen wrote:
> On Fri, 2014-02-14 at 06:23 +0100, David Henningsson wrote:
>> On 02/11/2014 03:03 PM, Tanu Kaskinen wrote:
>>> Hi all!
>>>
>>> I'm working (together with other people at Intel) on adding system
>>> volume control functionality to Tizen (primarily the IVI profile,
>>> although there's nothing inherently IVI specific here). Tizen uses web
>>> APIs for the UI, and since there are no existing web API standards for
>>> this functionality, we need to design the API from scratch. The current
>>> design can be seen at [1], in case you're interested (it's good
>>> background material for this discussion). What I'd like to discuss in
>>> this thread, however, is not the web API as such, but the use cases
>>> behind that design and what those use cases ultimately require from
>>> PulseAudio. (Feedback for the web API proposal is welcome too, but I
>>> think that should be discussed in a separate thread.)
>>>
>>> We want to support:
>>>  * A single main output volume control. Think of things like the Gnome
>>> volume applet: when you click the applet icon, you get a single volume
>>> slider for controlling the "current output", whatever that means. The
>>> Gnome volume applet implements the main volume by using the default sink
>>> volume, but we'd like to make the main volume contextual, so that the
>>> main volume can control different things at different times. Ideally the
>>> contextual logic would be implemented in PulseAudio (by a policy
>>> module), not in the volume applet. The volume applet should know when
>>> the main volume controls e.g. the headphones volume or the phone call
>>> volume.
>>>  * Volume classes. A car manufacturer may want to present only two
>>> volume controls for the user: "system" and "entertainment". A policy
>>> module in PulseAudio should classify streams based on whether their
>>> volume should be controlled by the "system" volume or by the
>>> "entertainment" volume. The volume classification should be separate
>>> from the routing classification (module-stream-restore, among other
>>> problems, doesn't allow separate classification for volume and routing).
>>>  * Fine-grained volume UIs that show all application and device volumes,
>>> like pavucontrol. PulseAudio already handles this pretty well.
>>>
>>> The first question: do you agree that PulseAudio should support those
>>> use cases in one way or another?
>>>
>>> If the answer to the first question is "yes", then we can start
>>> discussing the details of the implementation. How would you implement
>>> the support for the listed requirements?
>>>
>>> I propose that we introduce a new concept: volume control. It would be a
>>> separate object that could be referenced from other objects. For
>>> example, all devices would have their own volume control objects. The
>>> main volume could point to some of the device volume controls, or any
>>> other volume control object. Streams could have their own volume control
>>> objects, or multiple streams could reference the shared volume control
>>> of their volume class.
>>>
>>> introspect.h could have something like this:
>>>
>>> typedef struct pa_volume_control_info {
>>>     uint32_t index;
>>>     char *name;
>>>     char *description;
>>>     pa_volume_t volume;
>>>     pa_channel_map channel_map;
>>>     double *balance; /* per-channel balance, values between 0.0 and 1.0 */
>>> } pa_volume_control_info;
>>>
>>> The alternative for separate volume controls is that we keep embedding
>>> the volume in various other objects, like sinks, sources, ports (which,
>>> btw, are currently missing volume control altogether when they are
>>> inactive), sink inputs and source outputs. This is certainly a possible
>>> approach, but I very much prefer the separate volume control object idea.
>>>
>>> Benefits of separate volume control objects:
>>>  * Solves the problem of figuring out what the main volume controls. For
>>> example, the Gnome volume applet shows a headphone icon when the main
>>> volume controls the headphones volume. The volume applet can check if
>>> the main volume points to the same volume control object as the headset
>>> port.
>>>  * If two streams share the same volume, a pavucontrol-like application
>>> can group the two streams under one volume slider.
>>>  * Opportunity to separate the overall volume from the channel balance.
>>> The pa_cvolume volume representation loses all balance information when
>>> the user slides the overall volume to zero.
>>>  * Flexibility to easily add new volume control objects for whatever
>>> purpose in the future. For example, there could be a volume debug mode
>>> where the individual alsa volume elements would be exposed as read-only
>>> volume controls, or separate read-only volume control objects for the
>>> "reference", "real" and "soft" components of sink volumes. pactl could
>>> show and control all volumes without understanding what the volumes are
>>> associated with.
>>>
>>> Drawbacks of separate volume control objects:
>>>  * Duplication in the API. The old volume functionality in the public
>>> API can't be removed, so there would be two ways to control most volumes.
>>>
>>> The main volume would be added to pa_server_info as a new field (index
>>> referencing the appropriate volume control object).
>>>
>>> The volume classes would be separate objects:
>>>
>>> typedef struct pa_volume_class {
>>>     uint32_t index;
>>>     char *name;
>>>     char *description;
>>>     uint32_t volume_control;
>>> } pa_volume_class;
>>>
>>> I would add the volume control objects, the main volume and the volume
>>> classes to the core API, because this is functionality that is used
>>> pretty much everywhere (desktop environments, mobile devices, IVI
>>> systems), but I'm open to implementing this in a module too, with a
>>> protocol extension.
>>>
>>> [1] https://wiki.tizen.org/wiki/User:Tanuk/AudioSystemAPI
>>>
>>
>> Hi Tanu,
>>
>> my gut feeling would say:
>>
>> The volume objects, I think I understand the usefulness of those. I also
>> acknowledge the problem of having balance information lost when main
>> slider is zero.
> 
> Excellent!
> 
>> However we currently have the pa_cvolume type as well, which is sort of
>> the same as the new fields volume+channel_map+balance. I wonder if it
>> makes sense to group these together and call it pa_cvolume2 or something.
> 
> Does the grouping have any benefits? If pa_volume_control_info has a
> field with this pa_cvolume2 type, then it could be argued that for
> symmetry there should also be function
> pa_context_set_volume_control_volume() that takes a pa_cvolume2 struct
> as a parameter. However, when an application sets the attributes of a
> volume control object, usually it will only modify the overall volume or
> the balance, not both, and the channel map is immutable, so passing a
> pa_cvolume2 struct to the setter function is inconvenient.

A volume control UI with presets would typically set both. So would any
volume control UI do that does not look exactly the way you anticipate.

E g, pavucontrol has two sliders, one left and one right, and gnome
volume control has main volume + balance.

The grouping would also enable potential helper functions such as those
we have for pa_cvolume today.

> If we forget about the symmetry argument and pass something else than
> pa_cvolume2 to pa_context_set_volume_control_volume(), then the only
> purpose of pa_cvolume2 seems to be to reduce the number of fields in the
> pa_volume_control_info struct.
> 
>> And as somebody already pointed out, having either a global or
>> per-channel mute in that struct as well would probably be a good thing too.
> 
> I'd like to keep mute controls completely separate from volume. They are
> not inherently linked to each other (even though they very often appear
> together).

I don't understand this argument. A mute control *is* a volume control
(with two distinct values). In this case, it's a way to avoid volume
information lost on muted audio.
Well, and that everything said so far in this thread would apply equally
well to mute controls, so I guess you will come back in a few months and
want to add another core object for mute controls if we don't get it in
now ;-)

> Another question is that should mute controls have per-channel
> granularity. I don't remember anyone ever asking for that feature, and
> it would add complexity to the mute controls (clients need to deal with
> partially-muted controls, and we need to attach channel map information
> to the controls). On the other hand, it doesn't seem entirely pointless
> as a concept either.

Yeah, I'm not sure whether mute should be global or per channel either.
More opinions?

>> If you want main volume to represent one of these, you might want to add
>> a property list too, to contain the icon property (the UI could show a
>> headphone icon when controlling headphone volume).
> 
> Did this paragraph contain implied approval for adding main input and
> output volume control references to the pa_server_info struct?

Sure, we have default sink and source there already, so it seems to make
sense to me.

> What would you think about putting the icon name as a separate field to
> the pa_volume_control_info struct? I'm not against adding a proplist for
> undocumented metadata, but let's not abuse the proplist for things that
> are part of the documented stable API (any more than what we already
> do).

IIRC, icon names are in proplists everywhere else, so having them in a
proplist here as well would avoid inconsistency.

>> The volume classes however seem more like routing/policy implementation
>> internals. They could be kept inside the routing/policy module, exported
>> through property lists, or something like that. Either that, or I didn't
>> really understand the concept.
> 
> Applications should have a unified way to query what volume classes
> exist and what volume control they are associated with. An example of a
> volume class is the event volume class. If applications want to access
> that, they currently need to use a module-stream-restore specific API,
> even though the concept of an event volume class is not specific to
> module-stream-restore.

The concept "volume class" does not exist in module-stream-restore, at
least not with that name.

Could you give a clear definition of what "volume classes" are? If it's
just a way of grouping volume control objects, then volume control
objects could just have some property of what class they belong to.


-- 
David Henningsson, Canonical Ltd.
https://launchpad.net/~diwic