[pulseaudio-discuss] [RFC] Dynamically created PCM devices for HDMI/DisplayPort

[Raymond Yau]

>>
>> It looks like in the future the ALSA drivers for some Intel hardware
>> will dynamically create a new PCM device when a DisplayPort monitor is
>> plugged in. This is being discussed in this thread (part of the thread
>> is also cross-posted to pulseaudio-discuss):
>>
http://thread.gmane.org/gmane.comp.freedesktop.xorg.drivers.intel/62703/focus=63002
>>
>
> No objections to the plan below, but some nitpicks.
>
>
>>
>> PulseAudio doesn't currently support dynamic PCM devices, so work is
>> needed to add that support. I may work on that, or it may be someone
>> else from Intel. I'll describe here what code changes I think we should
>> make. That serves two purposes: to get feedback about the plan before
>> any code gets written, and to help with the implementation work if
>> someone else than me is going to write the code.
>>
>> This is a long mail, but I'm sure I still didn't think of every issue
>> that will arise when implementing this...
>>
>>
>> Event: monitor gets plugged in
>> ------------------------------
>>
>> The first thing that happens should be that PulseAudio gets a wakeup
>> from the alsa mixer, when a new ELD control for the monitor is added.
>> This is important, because the mixer should be ready when PulseAudio
>> starts to use the new PCM device. It's up to the driver developer to do
>> this right. This wakeup can be ignored, so no code changes are needed in
>> PulseAudio to handle this.
>>
>> The second thing that happens is that udev notifies PulseAudio about a
>> new PCM device. Interfacing with udev is done in
>> src/modules/module-udev-detect.c. Currently PulseAudio only cares about
>> new and removed cards, so module-udev-detect has to be modified to also
>> keep track of what PCM devices each card has, so that new devices can be
>> noticed.
>
>
> I am not sure whether this "wakeup from alsa mixer first, from udev
second" ordering is something that can be relied on. But a "ELD control
exists when udev notifies us" requirement looks sensible.
>
>
>>
>> When module-udev-detect sees a new PCM device, it needs to notify
>> module-alsa-card about it. module-udev-detect's only interface with
>> module-alsa-card is pa_module, which is not useful for adding the
>> notification. I think we should move the bulk of the code in
>> module-alsa-card.c to a new class: pa_alsa_card. module-udev-detect
>> would then create pa_alsa_card objects instead of loading
>> module-alsa-card instances. module-alsa-card would still exist as a
>> wrapper around pa_alsa_card, but the module would not be used by
>> module-udev-detect. With pa_alsa_card in place, we can add a
>> pa_alsa_card_pcm_added() function to its API.
>
>
> OK.
>
>
>> pa_alsa_card should be defined in src/modules/alsa/alsa-card.[ch] and
>> included in the libalsa-util.la helper library.
>>
>> When moving the code from module-alsa-card to pa_alsa_card, some changes
>> to the sink and source error handling is needed. Currently, if something
>> fails in the IO thread of an alsa sink or source, the sink/source
>> unloads the module that owns the sink/source (see the end of
>> thread_func() in alsa-sink.c and alsa-source.c). Now the owner module of
>> alsa sinks and sources becomes module-udev-detect, and we certainly
>> don't want to unload that if a single sink or source fails. The
>> sink/source should notify the pa_alsa_card object of the failure (new
>> functions pa_alsa_card_sink_failed() and pa_alsa_card_source_failed()),
>> and pa_alsa_card should free the failed pa_alsa_sink or pa_alsa_source
>> object.
>
>
> The case of manually-loaded module-alsa-sink is not described here, but
should be.
>
>
>> Let's get back to the pa_alsa_card_pcm_added() function. What should it
>> do? We shouldn't support dynamic PCMs for arbitrary hw PCMs, because
>> that's not compatible with relying on logical device names like "front",
>> "surround51" etc. At this point we only need to support dynamic PCMs
>> that are dedicated to hotplugged HDMI/DisplayPort/Thunderbolt devices.
>> The current proposal to detect such PCMs is to add a new HDMI class to
>> snd_pcm_class_t, which can be queried with snd_pcm_info_get_class().
>>
>> Currently when opening HDMI devices, we use "hdmi:x,y" as the device
>> string, where x is the card index and y is the device index. The device
>> index may be different than the hw device index, but the mapping between
>> "hdmi:x,y" to "hw:x,z" is static. The mapping can be different with
>> different drivers, AFAIK. We currently blindly try all device indexes
>> from 0 to 7 when probing the card. Takashi Iwai told that such behaviour
>> won't be compatible with drivers that create dynamic PCMs for HDMI. I
>> guess the reason is that the dynamically allocated hw device indexes can
>> (and usually do) fall outside the index range that is used in "hdmi:x,y".
>>
>> Since the new HDMI PCM class is new, old kernels and old alsa-lib won't
>> use that class even with PCMs that are actually dedicated to HDMI. We
>> need to tell apart drivers that use the new HDMI PCM class and drivers
>> that don't. With drivers that never use the HDMI class, we should keep
>> using the "hdmi:x,y" device strings. With drivers that use the HDMI
>> class, we should use "hdmi:CARD=x,SYSDEV=z", where z is the hw device
>> index (the SYSDEV parameter doesn't currently exist, so alsa-lib needs
>> to be updated to support it). How do we tell the two kinds of drivers
>> apart? The current proposal is to add a version field to the PCM info.
>> Version 0 would mean that the driver is unaware of the HDMI PCM class,
>> and version 1 would mean that the driver will set the PCM class to HDMI
>> when appropriate.
>>
>> I assume that the PCM info version will be provided separately for each
>> PCM device, but I expect the version to be always the same for every
>> device that belongs to the same card. We definitely need to make the
>> decision between the two models at the card level in any case, because
>> we can't really mix the "hdmi:x,y" model with the "hdmi:CARD=x,SYSDEV=z"
>> model within the same card. When probing a new card, we should check the
>> PCM info version of the first device that we probe, and choose the HDMI
>> model for the card based on that. If the first device has PCM info
>> version 0, then we won't support dynamic HDMI devices for that card,
>> even if subsequent devices would somehow have version 1.
>>
>> The "mapping" concept in our alsa-mixer code corresponds to the PCM
>> devices. What mappings exist is configured in
>> src/modules/alsa/mixer/profile-sets/default.conf. We have many HDMI
>> entries, here are a couple of examples:
>>
>> [Mapping hdmi-stereo]
>> description = Digital Stereo (HDMI)
>> device-strings = hdmi:%f
>> paths-output = hdmi-output-0
>> channel-map = left,right
>> priority = 4
>> direction = output
>>
>> [Mapping hdmi-stereo-extra1]
>> description = Digital Stereo (HDMI 2)
>> device-strings = hdmi:%f,1
>> paths-output = hdmi-output-1
>> channel-map = left,right
>> priority = 2
>> direction = output
>>
>> The "device-strings" option doesn't suit the HDMI case very well, if we
>> sometimes have to use "hdmi:x,y" and sometimes "hdmi:CARD=x,SYSDEV=z".
>> Also, the path configuration files assume a particular mapping from
>> "hdmi:x,y" to "hw:x,z" when dealing with ELD information and jack
>> detection, and that assumed mapping is incorrect with dynamic HDMI
>> devices. (This assumption probably also means that our HDMI ELD and jack
>> detection functionality is currently broken on non-HDA cards.)
>>
>> I propose that we add a new boolean option for mappings, which would
>> indicate that the mapping represents more than one PCM device, and that
>> the PCM and mixer handling is performed according to the complex HDMI
>> specific rules that I've explained above. The option name could be e.g.
>> "dynamic-hdmi". When "dynamic-hdmi" is be set for a mapping, the
>> "description", "device-strings", "paths-output" and "direction" options
>> in the configuration file would be ignored, and the appropriate values
>> for those would be hardcoded. That would allow us to shorten
>> default.conf quite a bit, and also remove all the hdmi-output-N.conf
>> path files (the generated paths would be hardcoded too).
>>
>> All that hardcoding isn't nice from flexibility point of view, but I
>> don't think the lost flexibility is very important in this case. I
>> believe that any alternative solution that would keep everything in the
>> configuration files would introduce a significant amount of complexity
>> to deal with the variance in the "hdmi:x,y" -> "hw:x,z" mapping (and I'm
>> not sure it's even possible to have non-HDA-specific ELD information and
>> jack detection support with the old-style "hdmi:x,y" device strings,
>> since we don't have a reliable method to figure out what hw PCM device
>> index y corresponds to).
>>
>> In case of dynamic HDMI mappings, the HDMI profiles need to become
>> dynamic too. So, if a profile definition in a configuration file
>> references a mapping that has the "dynamic-hdmi" flag set, that profile
>> definition will then represent multiple profiles, one for each HDMI
>> device. I'm not sure how to deal with the "description" option. Should
>> it be ignored, or should we append a number to the description when
>> there are multiple HDMI devices? When profiles are autogenerated (which
>> is the common case), then the existing profile description logic should
>> work fine.
>>
>> So, when pa_alsa_card_pcm_added() sees that a new HDMI PCM device
>> appeared, it should create a new mapping for the device, a new path for
>> the mapping, and a new profile containing the mapping. Then
>> pa_alsa_card_pcm_added() needs to call pa_card_add_profile(), and
>> hopefully it will just work.
>>
>>
>> Event: monitor gets unplugged
>> -----------------------------
>>
>> When a monitor gets unplugged, module-udev-detect gets a notified, and
>> it should figure out that a PCM device has disappeared. It should then
>> call pa_alsa_card_pcm_removed(), which is a new function that needs to
>> be implemented.
>
>
> Again, the case of manually loaded module-alsa-sink is not considered.
>
>
>>
>> pa_alsa_card_pcm_removed() should mirror pa_alsa_card_pcm_added(), just
>> undoing the things that _added() did, in reverse order. So, first it
>> should call pa_card_remove_profile(). That function doesn't exist
>> currently, so it needs to be implemented. Next
>> pa_alsa_card_pcm_removed() should free the pa_alsa_profile,
>> pa_alsa_mapping and pa_alsa_path objects corresponding to the removed
>> device. I think that's it for pa_alsa_card_pcm_removed().
>>
>> If the profile that is being removed is active, pa_card_remove_profile()
>> should change the card profile to something else. I suppose it should
>> use the same logic as what pa_card_new() uses when choosing the default
>> profile.
>>
>
> What else is missing is the big picture of the available multi-monitor
use cases.
>
> Currently, if one has a card with multiple HDMI outputs, one can use
audio from one monitor at a time, using profiles. I.e. there are no
profiles like "HDMI Digital Stereo Output + HDMI 3 Digital Surround Output"
(with the possibility to move individual streams between monitors),
presumably due to the combinatorial explosion. Will this limitation be
lifted?

>
> Also, AFAIR, there was a discussion of a possibility to send the same PCM
stream to two monitors. How would that work?

http://www.intel.com/support/graphics/sb/CS-031040.htm

The multi display monitors can be configured by graphic driver in clone
mode or extended display mode

Are there hardware limitation (e.g. bandwidth ) which limit the highest
sample rate and 32bits cannot be used by multi monitors at same time ?

Are there any different in primary display , second display and third
display ?

Will th power off/on of the monitor in the middle of dasiy chain affect the
other displays of the dasiy chain ?

http://www.intel.com/support/graphics/sb/CS-033714.htm

How do pulseaudio/driver know multi displays are in clone node or
independent display mode ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/pulseaudio-discuss/attachments/20150701/efdf3974/attachment.html>