[pulseaudio-discuss] RFC: Routing and Priority lists

[Colin Guthrie]

Hi,

Sorry for the late reply.

'Twas brillig, and Janos Kovacs at 16/11/11 13:04 did gyre and gimble:
> I try to build a multimedia policy for embeded systems like smart
> phones, TV's in vehicle entertainment systems etc.
> The idea is to have a policy decsion making daemon that would control
> audio/video routing among other things. This policy
> daemon supposed to push down policies to the actual madia handling
> daemons eg. pulseaudio, X server or Wayland
> compositor. So my comments come from that angle.

Many thanks for the insight on your background.

> In general the priority list seem to be suitable approach for us.
> However, in the routing decision we want to deal with logical
> devices like 'speaker', 'headset'  etc not cards, sinks ports etc.

Yes, this should generally be OK. The entries in each priority list are
what I've coined as "device+ports". This is basically a specific port of
a given device.

e.g. on some HW I have, I have a single sink, but that sink consists of
two ports: Speaker and Headphones.

In a given priority list I may favour the built in Headphones first,
then some external USB headset, then the Speakers.

So as you can see the single sink actually fits into the priority list
in two different places (or as many different places as there are ports).

Some separate logic (ultimately coming from David's work on jack
detection) would ensure a given port is activated when it becomes
"available". This means that I have some stream playing and it's on the
speakers, I plugin my USB headset and the stream moves over
automatically (due to a "reroute" event triggered when the new sink
becomes available). Then I plug in my headphones into the 3.5mm jack
socket, and it's detected as now being "available". The "separate logic"
mentioned, switches port from "speakers" to "headphones" and a "reroute"
event is triggered (just like a new sink appearing/disappearing,
"reroute" events are triggered on port changes too) and the stream is
automatically moved to the Headphones.

Hope this small example highlights how things will work.

NB I've used the term "separate logic" here quite loosely. David has
some code to do this already, but this may eventually be superceded by
logic inside the routing daemon that will automatically switch to higher
priority ports on sinks if they are available. This is, however, perhaps
not something that is always desirable. e.g. on the recording side, both
Mic-in and Line-in might be different ports and both might be available.
Simple switching when available might not be best covered here. Also, as
noted in the wiki article, the auto-switching mode might result in
different routing decisions depending on the order of streams... I won't
elaborate too much here as I've covered this potential problem on the
wiki already.

> That would allow to separate policies from the HW adaptation.
> IMO the mapping of logical devices to actual cards/sinks/sources/ports
> should be done in pulseaudio. One reason we are
> interested in UCM is that UCM natively support those logical devices.

Tanu already covered this point but this shouldn't IMO be a specific
issue here.

> The routing is somewhat similar what Tanu once proposed some time ago.
> Generally the routing would happen like:
> 1.) the policy decision point would figure out a priority list of
> logical devices for each media roles and push those lists to
>      pulseaudio.

OK, that fits well.

> 2.) a routing module in pulseaudio would maintain mappings between
> logical devices and actual cards/sinks/sources/ports
>      and the availablility of those logical devices. In case of
> multiple logical devices it would select the preferred one (lets
>      discuss later how this would happen).

OK, so I think this is partially covered by the "separate logic"
mentioned above. Automatically activating ports (e.g. Headphones) when a
jack is plugged in will be possible and then automatic reroute should
honour any priority list preferences related to favouring headphones
over e.g. bluetooth devices etc.

> 3.) the routing module would also track the streams and maintain a
> list of the active media roles.

OK, I won't specifically do that in the proposed implementation, but
(and feel free to correct me if I'm wrong) this is just a simple matter
of iterating over the sink inputs and check the media.role property on
each of them.

> 4.) when streams appear/disappear the the routing module would figure
> out the card and port settings by walking through
>      of the active media roles. Media roles would have priorities and
> the routing module would first set the cards/ports for the
>      highest priority routing target of the highest priority media
> role. The subsequent media roles would be routed to the first
>      non-conflicting routing target (that can be in worst case the
> nulll sink). Consequently the cards/ports would be set
>      accordingly.

OK, this is interesting and just the kind of feedback I was hoping for :)

In this case the NB mentioned above for my "separate logic" comment has
some quirks. This approach (of saying some roles are "more important"
than others) actually solves the problem of an "auto switch" mode doing
things differently depending on the order of stream start.

I'll try and bash this out a bit more when designing any auto switching
mode.

> 5.) the actual streams will be (re)routed to their sinks/sources if needed.

Cool.

> If the routing infrastructure will not support logical devices as
> routing targets, in step 4 the routing module would create
> for each media role a routing list with the single sink entry.
> 
> I am actually prototyping something to see how this algorithm would
> work in practice. What we call a logical device maps
> to device port in pulseaudio. It seems that the above algorithm can be
> done but some infrastructural support for logical devices
> would be good. For instance adding properties to device ports would
> help.

Interestingly, I suggested to David that ports ultimately have a
priority list attached. My use case was primarily for icons in UIs but
the generic us of a proplist was advised for consistency.

Having different ports with different "intended roles" was also
something that might be useful for UCM so this is all beginning to fall
into place I think.

> A property that describes the connected device
> would help a lot (ie. instead of 'analog output' one could see that
> actually a 'speaker' is connected to the jack). And of cource
> some protocol change that this property could be get/set by applications.

That might be useful (i.e. knowing what was connected to the jack) but I
think this could be getting a bit too low level. I think attaching the
intended-role is more generic but can ultimately be used for the same thing.

> I think the pulseaudio routing module and the policy daemon would not
> be very much needed in desktops but makes much
> sense IMO in embedded systems with limited user interaction capabilities.

Yeah I agree in principle, but actually a lot of the uses cases do
overlap and while I'm keen to avoid unnecessary complexity in the
implementation I'm planning, if it only gets a little more complex and
suddenly becomes universally useful, then IMO it's worth that extra
complexity!

> My primary objective is to make sure that upcoming pulseaudio routing
> infrastructural, port detection and UCM support
> related changes are not blocking the above described algorithm. I also
> hope to attract some other folks to this idea to
> use the policy daemon + the pulseaudio module in embedded systems.
> 
> First I am interested in your opinions and go into details if you were
> interested in such approach.

Great! I think this very much fits in but your point 5 is certainly food
for thought but I think I just need to store just a small amount of
extra metadata in my proposed structure to allow that kind of decision
to be made. Essentially (to use the generic terminology I'm using on the
wiki) I think I need to store an additional "weight" with each key in a
priority list.


I'll try and update the wiki in the coming days with a slightly
different API to accommodate this extra weight and of course write it up
a bit more to highlight your use case.


I'll reply to some of the points in Tanu's email now, but will try and
keep it short :)

Cheers


Col


-- 

Colin Guthrie
gmane(at)colin.guthr.ie
http://colin.guthr.ie/

Day Job:
  Tribalogic Limited http://www.tribalogic.net/
Open Source:
  Mageia Contributor http://www.mageia.org/
  PulseAudio Hacker http://www.pulseaudio.org/
  Trac Hacker http://trac.edgewall.org/