[pulseaudio-discuss] [RFC] Dynamically created PCM devices for HDMI/DisplayPort

[Alexander E. Patrakov]

30.06.2015 20:08, Tanu Kaskinen wrote:
> Hi all,
>
> It looks like in the future the ALSA drivers for some Intel hardware
> will dynamically create a new PCM device when a DisplayPort monitor is
> plugged in. This is being discussed in this thread (part of the thread
> is also cross-posted to pulseaudio-discuss):
> http://thread.gmane.org/gmane.comp.freedesktop.xorg.drivers.intel/62703/focus=63002
>

No objections to the plan below, but some nitpicks.

>
> PulseAudio doesn't currently support dynamic PCM devices, so work is
> needed to add that support. I may work on that, or it may be someone
> else from Intel. I'll describe here what code changes I think we should
> make. That serves two purposes: to get feedback about the plan before
> any code gets written, and to help with the implementation work if
> someone else than me is going to write the code.
>
> This is a long mail, but I'm sure I still didn't think of every issue
> that will arise when implementing this...
>
>
> Event: monitor gets plugged in
> ------------------------------
>
> The first thing that happens should be that PulseAudio gets a wakeup
> from the alsa mixer, when a new ELD control for the monitor is added.
> This is important, because the mixer should be ready when PulseAudio
> starts to use the new PCM device. It's up to the driver developer to do
> this right. This wakeup can be ignored, so no code changes are needed in
> PulseAudio to handle this.
>
> The second thing that happens is that udev notifies PulseAudio about a
> new PCM device. Interfacing with udev is done in
> src/modules/module-udev-detect.c. Currently PulseAudio only cares about
> new and removed cards, so module-udev-detect has to be modified to also
> keep track of what PCM devices each card has, so that new devices can be
> noticed.

I am not sure whether this "wakeup from alsa mixer first, from udev 
second" ordering is something that can be relied on. But a "ELD control 
exists when udev notifies us" requirement looks sensible.

>
> When module-udev-detect sees a new PCM device, it needs to notify
> module-alsa-card about it. module-udev-detect's only interface with
> module-alsa-card is pa_module, which is not useful for adding the
> notification. I think we should move the bulk of the code in
> module-alsa-card.c to a new class: pa_alsa_card. module-udev-detect
> would then create pa_alsa_card objects instead of loading
> module-alsa-card instances. module-alsa-card would still exist as a
> wrapper around pa_alsa_card, but the module would not be used by
> module-udev-detect. With pa_alsa_card in place, we can add a
> pa_alsa_card_pcm_added() function to its API.

OK.

> pa_alsa_card should be defined in src/modules/alsa/alsa-card.[ch] and
> included in the libalsa-util.la helper library.
>
> When moving the code from module-alsa-card to pa_alsa_card, some changes
> to the sink and source error handling is needed. Currently, if something
> fails in the IO thread of an alsa sink or source, the sink/source
> unloads the module that owns the sink/source (see the end of
> thread_func() in alsa-sink.c and alsa-source.c). Now the owner module of
> alsa sinks and sources becomes module-udev-detect, and we certainly
> don't want to unload that if a single sink or source fails. The
> sink/source should notify the pa_alsa_card object of the failure (new
> functions pa_alsa_card_sink_failed() and pa_alsa_card_source_failed()),
> and pa_alsa_card should free the failed pa_alsa_sink or pa_alsa_source
> object.

The case of manually-loaded module-alsa-sink is not described here, but 
should be.

> Let's get back to the pa_alsa_card_pcm_added() function. What should it
> do? We shouldn't support dynamic PCMs for arbitrary hw PCMs, because
> that's not compatible with relying on logical device names like "front",
> "surround51" etc. At this point we only need to support dynamic PCMs
> that are dedicated to hotplugged HDMI/DisplayPort/Thunderbolt devices.
> The current proposal to detect such PCMs is to add a new HDMI class to
> snd_pcm_class_t, which can be queried with snd_pcm_info_get_class().
>
> Currently when opening HDMI devices, we use "hdmi:x,y" as the device
> string, where x is the card index and y is the device index. The device
> index may be different than the hw device index, but the mapping between
> "hdmi:x,y" to "hw:x,z" is static. The mapping can be different with
> different drivers, AFAIK. We currently blindly try all device indexes
> from 0 to 7 when probing the card. Takashi Iwai told that such behaviour
> won't be compatible with drivers that create dynamic PCMs for HDMI. I
> guess the reason is that the dynamically allocated hw device indexes can
> (and usually do) fall outside the index range that is used in "hdmi:x,y".
>
> Since the new HDMI PCM class is new, old kernels and old alsa-lib won't
> use that class even with PCMs that are actually dedicated to HDMI. We
> need to tell apart drivers that use the new HDMI PCM class and drivers
> that don't. With drivers that never use the HDMI class, we should keep
> using the "hdmi:x,y" device strings. With drivers that use the HDMI
> class, we should use "hdmi:CARD=x,SYSDEV=z", where z is the hw device
> index (the SYSDEV parameter doesn't currently exist, so alsa-lib needs
> to be updated to support it). How do we tell the two kinds of drivers
> apart? The current proposal is to add a version field to the PCM info.
> Version 0 would mean that the driver is unaware of the HDMI PCM class,
> and version 1 would mean that the driver will set the PCM class to HDMI
> when appropriate.
>
> I assume that the PCM info version will be provided separately for each
> PCM device, but I expect the version to be always the same for every
> device that belongs to the same card. We definitely need to make the
> decision between the two models at the card level in any case, because
> we can't really mix the "hdmi:x,y" model with the "hdmi:CARD=x,SYSDEV=z"
> model within the same card. When probing a new card, we should check the
> PCM info version of the first device that we probe, and choose the HDMI
> model for the card based on that. If the first device has PCM info
> version 0, then we won't support dynamic HDMI devices for that card,
> even if subsequent devices would somehow have version 1.
>
> The "mapping" concept in our alsa-mixer code corresponds to the PCM
> devices. What mappings exist is configured in
> src/modules/alsa/mixer/profile-sets/default.conf. We have many HDMI
> entries, here are a couple of examples:
>
> [Mapping hdmi-stereo]
> description = Digital Stereo (HDMI)
> device-strings = hdmi:%f
> paths-output = hdmi-output-0
> channel-map = left,right
> priority = 4
> direction = output
>
> [Mapping hdmi-stereo-extra1]
> description = Digital Stereo (HDMI 2)
> device-strings = hdmi:%f,1
> paths-output = hdmi-output-1
> channel-map = left,right
> priority = 2
> direction = output
>
> The "device-strings" option doesn't suit the HDMI case very well, if we
> sometimes have to use "hdmi:x,y" and sometimes "hdmi:CARD=x,SYSDEV=z".
> Also, the path configuration files assume a particular mapping from
> "hdmi:x,y" to "hw:x,z" when dealing with ELD information and jack
> detection, and that assumed mapping is incorrect with dynamic HDMI
> devices. (This assumption probably also means that our HDMI ELD and jack
> detection functionality is currently broken on non-HDA cards.)
>
> I propose that we add a new boolean option for mappings, which would
> indicate that the mapping represents more than one PCM device, and that
> the PCM and mixer handling is performed according to the complex HDMI
> specific rules that I've explained above. The option name could be e.g.
> "dynamic-hdmi". When "dynamic-hdmi" is be set for a mapping, the
> "description", "device-strings", "paths-output" and "direction" options
> in the configuration file would be ignored, and the appropriate values
> for those would be hardcoded. That would allow us to shorten
> default.conf quite a bit, and also remove all the hdmi-output-N.conf
> path files (the generated paths would be hardcoded too).
>
> All that hardcoding isn't nice from flexibility point of view, but I
> don't think the lost flexibility is very important in this case. I
> believe that any alternative solution that would keep everything in the
> configuration files would introduce a significant amount of complexity
> to deal with the variance in the "hdmi:x,y" -> "hw:x,z" mapping (and I'm
> not sure it's even possible to have non-HDA-specific ELD information and
> jack detection support with the old-style "hdmi:x,y" device strings,
> since we don't have a reliable method to figure out what hw PCM device
> index y corresponds to).
>
> In case of dynamic HDMI mappings, the HDMI profiles need to become
> dynamic too. So, if a profile definition in a configuration file
> references a mapping that has the "dynamic-hdmi" flag set, that profile
> definition will then represent multiple profiles, one for each HDMI
> device. I'm not sure how to deal with the "description" option. Should
> it be ignored, or should we append a number to the description when
> there are multiple HDMI devices? When profiles are autogenerated (which
> is the common case), then the existing profile description logic should
> work fine.
>
> So, when pa_alsa_card_pcm_added() sees that a new HDMI PCM device
> appeared, it should create a new mapping for the device, a new path for
> the mapping, and a new profile containing the mapping. Then
> pa_alsa_card_pcm_added() needs to call pa_card_add_profile(), and
> hopefully it will just work.
>
>
> Event: monitor gets unplugged
> -----------------------------
>
> When a monitor gets unplugged, module-udev-detect gets a notified, and
> it should figure out that a PCM device has disappeared. It should then
> call pa_alsa_card_pcm_removed(), which is a new function that needs to
> be implemented.

Again, the case of manually loaded module-alsa-sink is not considered.

>
> pa_alsa_card_pcm_removed() should mirror pa_alsa_card_pcm_added(), just
> undoing the things that _added() did, in reverse order. So, first it
> should call pa_card_remove_profile(). That function doesn't exist
> currently, so it needs to be implemented. Next
> pa_alsa_card_pcm_removed() should free the pa_alsa_profile,
> pa_alsa_mapping and pa_alsa_path objects corresponding to the removed
> device. I think that's it for pa_alsa_card_pcm_removed().
>
> If the profile that is being removed is active, pa_card_remove_profile()
> should change the card profile to something else. I suppose it should
> use the same logic as what pa_card_new() uses when choosing the default
> profile.
>

What else is missing is the big picture of the available multi-monitor 
use cases.

Currently, if one has a card with multiple HDMI outputs, one can use 
audio from one monitor at a time, using profiles. I.e. there are no 
profiles like "HDMI Digital Stereo Output + HDMI 3 Digital Surround 
Output" (with the possibility to move individual streams between 
monitors), presumably due to the combinatorial explosion. Will this 
limitation be lifted?

Also, AFAIR, there was a discussion of a possibility to send the same 
PCM stream to two monitors. How would that work?

-- 
Alexander E. Patrakov