[pulseaudio-discuss] RFC: New volume functionality for PulseAudio

[Tanu Kaskinen]

On Fri, 2014-06-27 at 14:41 +0200, David Henningsson wrote:
> 
> On 2014-06-25 11:47, Tanu Kaskinen wrote:
> > On Tue, 2014-06-24 at 15:37 +0200, David Henningsson wrote:
> >>
> >> On 2014-06-24 15:01, Tanu Kaskinen wrote:
> >>> On Tue, 2014-06-24 at 13:19 +0200, David Henningsson wrote:
> >>>> I remember thinking that pa_ext_volume_api_bvolume should include a "int
> >>>> mute" field. Then every mute specific stuff, including mute controls,
> >>>> can be skipped.
> >>>
> >>> I don't like that. Any struct that contains both volume and mute will
> >>> also need "bool has_volume" and "bool has_mute" fields (or a type enum
> >>> for either/or choice).
> >>
> >> Wouldn't be typical for a control to have both volume and mute? E g, a
> >> sink has both volume and mute, right? Instead of having two objects and
> >> leave to the client to group them together, it would make more sense to
> >> keep them as one object and let clients turn it into two if necessary.
> >>
> >> Or, why wouldn't a volume also have a mute?
> >
> > If we route things in domains where pulseaudio doesn't have access to
> > the audio data, we're at the mercy of whatever controls the domain
> > provides. For example, when routing audio from cellular modem to
> > speakers without ever seeing the audio data in pulseaudio, there's no
> > guarantee that the hardware has both volume element and a mute switch
> > for the audio path. It can have one or the other. I haven't worked with
> > such systems, though, so I don't know what the situation is with real
> > hardware out there.
> 
> Well, with that argument we could just as well use pa_cvolume. Because 
> we don't know if the hardware maintains balance when main volume is -inf 
> either.

It's enough if we remember in pulseaudio the accurate balance that was
set before the overall volume changed to -inf. The hardware doesn't need
to (and can't) remember it. (Btw, I prefer term "overall volume" when
referring to pa_bvolume.volume, because "main volume" is already used to
mean a different thing.)

The balance information is potentially lost only if we copy the volume
from the hardware to our own volume representation, but that's rarely
necessary. Usually we copy the volume from our own representation to the
hardware, not the other way around.

> We decided to have an extra "main" volume not to lose balance 
> information, we can have an extra mute not to lose "main" volume when 
> the stream/sink/etc is muted.

Losing the overall volume information is not a problem if volume and
mute are in separate control objects, so I fail to see the point you're
trying to make here.

> > It can also be that e.g. a stream should have both volume and mute, but
> > those control entirely different things. For example, someone once
> > requested a feature where the stream volume would actually control the
> > sink volume. I don't think that was a silly request: I might want to use
> > such feature myself for selected streams: for example, I might want the
> > volume control in Totem to control the device volume to get some of the
> > benefits of flat volumes (easy access to full hw volume range in
> > full-screen mode) without enabling flat volumes for all streams. If
> > stream volume actually controls the device volume, it may still be
> > desirable to have per-stream mute, so different controls are required
> > for volume and mute.
> >
> > I'll quote Alexander[1] for another use case for volume and mute
> > separation: "Sooner or later, someone will implement optional and
> > runtime-switchable automatic gain control (AGC). A natural question is:
> > does the whole notion of the main input volume make sense at all if AGC
> > is enabled?" Main input volume doesn't make sense in this case, but main
> > input mute does.
> >
> > [1] http://article.gmane.org/gmane.comp.audio.pulseaudio.general/19694/
> 
> Right, and compressed audio is another case where we can do sw mute, but 
> not sw volume.
> 
> >> We've already added extra information in order not to lose the balance
> >> when the volume is -inf, we could just as well add a mute too.
> >>
> >>> I don't think pa_bvolume is the right struct to
> >>> do that. This would make more sense to me:
> >>>
> >>> struct pa_control_info {
> >>>       uint32_t index;
> >>>       char *name;
> >>>       char *description;
> >>>       pa_proplist *proplist;
> >>>       pa_control_type_t type;
> >>>       void *data;
> >>> };
> 
> Do you also plan to add stream and sink pointers to two pa_control_info 
> structs then, one for volume and one for mute? I e for introspection of 
> sinks/streams/etc.

Well, the short answer is "no, I don't plan to do that". But the
question made two assumptions that suggest some misunderstanding, so I
should elaborate.

The first assumption is that one volume control object is associated
with only one sink/stream/etc. Multiple streams (or other kind of
entities) can share a common volume control, so a single pointer isn't
enough. Adding an array of pointers to all streams that the control is
associated with is an option that can be considered, but I haven't
planned to do that.

That said, having only one sink/stream/etc association per control is a
valid design that can be discussed if you like it. Since yesterday, I
have some unexpected trouble with sharing stream volume controls anyway,
due to flat volumes.

The second "odd" assumption was that having pointers from controls to
sinks/streams/etc would be somehow relevant for sink/stream/etc
introspection. If you want to introspect a sink, you need a pointer from
the sink to the control, not the other way around.

Now, some kind of mechanism to figure out the "primary" purpose of a
volume control object is needed, if a UI wants to indicate that the main
volume currently controls the call volume, or the headphones device, for
example. I initially thought that it's enough to check what stream,
device, or audio group is using the same volume control as what is
assigned as the main volume control, but this is ambiguous if multiple
entities share the volume control.

> >>> type is either PA_CONTROL_TYPE_VOLUME or PA_CONTROL_TYPE_MUTE. data
> >>> points to one of these structs depending on the control type:
> >>>
> >>> struct pa_volume_control_data {
> >>>       pa_bvolume volume;
> >>>       int convertible_to_dB;
> >>
> >> Btw, could you explain what this field (convertible_to_dB) means?
> >
> > Sinks and sources have the DECIBEL_VOLUME flag. That's exposed here, so
> > that clients know when they can convert pa_volume_t to decibels.
> 
> So in short, expect convertible_to_dB to always be true, if it's false, 
> we're dealing with "bad" hardware that does not know what their volume 
> steps mean?

Yes. (By the way, all Bluetooth headsets are "bad" in this sense too.)

> We could do the full capabilities thing that ALSA does - tell exactly 
> what steps and so on that the hw supports. In which case we can merge 
> volume and mute - a mute control is just a volume control with two 
> steps. But this does not seem to be what you're planning.

Steps are indeed something that probably should be added to volume
controls at some point, to allow UIs to show the user what steps are
available. Another use case for volume steps is to limit the dynamic
range by using artificial volume steps and setting the lowest non-mute
step to relatively high value. This is useful if the device has low
dynamic range and is too quiet on lower volumes. There might be some
better solution for the dynamic range problem than forcing the use of
artificial volume steps, but I've heard that Sailfish is using volume
steps for this purpose (via a custom volume API).

I'm not sure what design you're thinking here. Earlier you suggested a
design where a device would have one control that has both volume and
mute information. Is this paragraph about a design where a device would
have two volume controls, one of which actually represents mute with a
two-step volume slider? Another possibility is that you're thinking a
design where a device has just one merged volume/mute control without
separate mute field, but that's not workable because then we couldn't
remember the previous volume when the device is muted.

> I can understand that mute and volume are the most common types of 
> (volume) controls. But conceptually, we have everything in between too: 
> Per channel mutes (common in hw and ALSA), volume with a limited number 
> of steps, and so on. Considering this, and the boundary between volume 
> and mute is fluid.
> But maybe that is just to over-generalising things and we should stick 
> to volumes and mutes, like you suggest.

I'll try to summarize:

Bundling volume and mute into one control is not feasible if we want to
have flexible control sharing (see my example of sharing the device
volume control with Totem's stream, while also having per-stream mute
controls), and it's at least awkward in situations where only mute is
relevant (e.g. passthrough streams). For these reasons I want to
separate volume and mute into separate control objects.

Another idea was to represent mute with a volume control, so each device
would have two volume controls, one of which would be a two-step volume
that emulates mute. That should work, but I think that would look
strange to application developers, and I don't think it would save much
implementation effort if volume and mute controls are both based on a
common pa_control class anyway.

Then there's the possibility that we drop the control sharing feature.
This makes some issues go away, with the downside that we lose the
transparency in how different volumes relate to each other, so if we
want to describe in the client API how stream volumes and mutes are
grouped, some other mechanism is needed in place of control object
sharing.

I briefly mentioned my problem with flat volumes earlier, I'll explain
it a bit more here. In flat volume mode, the idea is that clients see
the absolute volume of the stream, i.e. the full hardware volume range
is available through the stream volume control. If a stream volume is
shared with an audio group, this becomes a problem, because the audio
group's volume control can't represent the absolute volume, because
there can be multiple streams connected to multiple devices - some
devices may use flat volume and some not, but even if all devices use
flat volume, their volume scales are different, so absolute volume means
different things on different devices. Therefore, the only sane choice
is that the audio group volume represents the relative stream volume.
But then, if a stream shares the volume control with an audio group, the
clients see the audio group's volume as the stream volume, so the
requirement of providing the full hw volume range is not met.

The solution is to create a volume control object for providing access
to the stream absolute volume, and propagate changes to that volume
control internally to the audio group's volume control. But then
transparency is lost, i.e. clients don't see the connection between the
stream volume and the audio group volume, which I wanted to be visible.
Since the volume control sharing doesn't work in a common case like
this, I don't see it as a big loss if we add the constraint that one
volume control can be associated with only one entity. It probably makes
some things simpler.

-- 
Tanu