[pulseaudio-discuss] Making phonon/kde work nicely with pulse

[Sean McNamara]

On Tue, Feb 24, 2009 at 10:39 PM, Lennart Poettering
<lennart at poettering.net> wrote:
> On Tue, 24.02.09 21:20, Colin Guthrie (gmane at colin.guthr.ie) wrote:
>
>> Hi Lennart,
>>
>> OK, so attached should be a screenie of the KDE settings window.
>>
>> As you can see, the idea is to list all the various detected h/w and
>> allow them to be prioritised accordingly. The categories are shown on
>> the left hand side, but each once shows the same set of "devices"
>> (albeit potentially in different orders!)
>
> I think this is both too simple (i.e. ignore system state, docking
> station, see below) and too complex (the preference list should be
> built automatically and can be built automatically I believe).
>
>> As is clear it does show some devices that are not currently
>> attached/detected.
>>
>> So to make this work nicely with pulse, I'd need to be able to:
>>  1. Record the sinks seen and know whether they are currently alive or
>> not (e.g. for USB, bluetooth and network sinks).
>>  2. Provide a way to prioritise our sinks in a list. If the most
>> preferred sink is not available (and our stream is not specifically
>> playing on a given sink) it should automatically pick the next device
>> with the highest priority.
>>  3. Allow this priority list to be different for different
>> roles/categories.
>> or
>>  3. Find a way to have something at the phonon side control the routing
>> decisions in PA (e.g. by uploading rules etc)
>
> There's already an API for this. See pulse/ext-stream-restore.h
>
>> What do you think is needed and what should I work on to get there? Do
>> you see any major roadblocks?
>
> Hmm, when we do this: the profile stuff probably needs to kept in mind
> as well: i.e. it might make sense to switch the profile a card in some
> cases by m-s-r. I.e. if a stream is tagged as "phone" call we should
> switch to HSP/HFP mode of a bt headset. If it is tagged as "music" we
> should switch to A2DP mode. This is obviously a pretty static rule,
> but for ALSA devices it might be more dynamic/history based. i.e. for
> "phone" switch to the internl speaker profile of an alsa card, for
> "video" and "music" to the spdif profile, and so on.
>
> And then there's also the big question what to do about jack sensing
> and docking station support. In both cases the profile of a card might
> need to be switched, and streams might need to connect to a different
> sink.
>
> I haven't yet gotten my mind all around this.
>
> If we beef this up, then we should it properly, i.e. somehow bring
> profiles and jack sensing/docking station support into the
> equation... No cluet how that should look like however.
>
> To formalize this a bit: The decision of selecting the appropriate
> device/profile for a stream should be based on multiple variables:
>
> - properties of the stream itself
>
> - system state (docked/undocked)
>
> - card state (jack sensing)
>
> - history

As discussed on IRC, this problem becomes more difficult with the
addition of network sinks. Remember, one of PA's main features is its
network transparency, so let's not ignore that :)

What we've assumed so far is that we have notifications from each sink
when (a) it goes from available -> unavailable, and (b) it goes from
unavailable -> available.

This is nice and easy for HAL, jack sensing, etc. Well, relatively
easy: we can hook into callbacks at the appropriate layer because this
stuff is monitored by lower levels of the system.

For networking, we only know about the case (a). We really have no way
of registering for notification of when a network-backed sink goes
from being unavailable to available. It seems that, to resolve that
problem, you would either have to poll, or add an entirely new class
of functionality: having a PA daemon become aware of clients that are
coming in from module-tunnel-sink, and send a "Hi, I'm alive" packet
to clients when the daemon starts.

On a local system, notifications, frequent polling and/or availability
notifications are fairly standard and inexpensive. But over the
network, we have firewall rules, angry network administrators, router
restrictions, NAT, etc. which complicate things. If you want to have
network-backed sinks in your priority list _at all_, you'll have to
address this some way. For instance, if I'm playing sound through a
module-tunnel-sink and the remote PA server goes down, at what point
(and through what mechanism) would we know to make that an "active"
choice in the list again?

In other words, solving the generic problem of "determining the
instantaneous availability state of an arbitrary sink" is logically
prior to building your priority list. And unfortunately, the
availability logic has to be hand-coded for each different class of
sink. It gets even worse if you think about the availability of
virtual sinks, such as combined sinks.

I haven't studied the innards of PA enough: do we have functionality,
at the generic sink level, of registering for notifications on when a
sink has become available or unavailable? If not, it would seem that
the first step to realizing this functionality at the Pulse<->Phonon
level is to submit a feature request to PA to implement the hairy
details of what we're discussing (or to generalize existing
availability logic for e.g. HAL-backed cards to the generic case of a
sink).

Also, please don't just think of KDE when implementing this. The more
of the "priority list" logic and "availability notification" logic you
can implement PA-side, the easier it would be to hook up a GUI to this
same functionality on another desktop environment, perhaps one not
using Phonon.

So, basically: With the concept of "card" in PA, we can cover all the
bases of physically connected devices such as USB adapters, docking
station speakers, headphones, and probably even Bluetooth; but many
useful audio paths through PA do not have an associated notion of
"card". So we must think in terms of the first-class audio path object
as being the sink (and we can apply the same logic to sources for
input). All of the features needed for the priority list must be
considered for arbitrary sinks, and we should design the feature set
in a way that (at least) allows any existing module-*-sink code to be
extended with notification of availability/unavailability logic.

I do see the problem with jack sensing, though: due to the way jacks
are registered in ALSA as being part of the same card, we are tempted
to intermingle the state of a "card" and the state of a "sink". Since
module-alsa-sink is essentially an audio channel to a single ALSA pcm
(e.g. hw:0), we would need a different sink if we wanted to specify a
specific jack, like HDMI instead of audio-out. And who's to decide
whether the user wants HDMI or analog audio-out if they're both
plugged in, anyway? Shouldn't the user be able to select? :)

This is a messy topic. It sounds like something Lennart should write a
come-to-the-rescue blog about! I thought the one about glitch-free was
great, as well as the sound API guide. We need some clear thinking and
a master plan on how to do this robustly; I don't think I'm being that
kind of clear in this post, but I know you can, Lennart.

Thanks,

Sean

>
> Part of this ruleset needs to be static. Part of it dynamically and
> bound to history.
>
> Gah, this becomes messy.
>
> Lennart
>
> --
> Lennart Poettering                        Red Hat, Inc.
> lennart [at] poettering [dot] net         ICQ# 11060553
> http://0pointer.net/lennart/           GnuPG 0x1A015CC4
> _______________________________________________
> pulseaudio-discuss mailing list
> pulseaudio-discuss at mail.0pointer.de
> https://tango.0pointer.de/mailman/listinfo/pulseaudio-discuss
>