[pulseaudio-discuss] [Intel-gfx] [PATCH 3/4] snd: add support for displayport multi-stream to hda codec.

[Takashi Iwai]

At Tue, 23 Jun 2015 07:51:22 +0000,
Kaskinen, Tanu wrote:
> 
> (Added pulseaudio-discuss to CC.)
> 
> On Mon, 2015-06-22 at 17:44 +0200, Takashi Iwai wrote:
> > At Mon, 22 Jun 2015 15:21:16 +0000,
> > Kaskinen, Tanu wrote:
> > > 
> > > On Mon, 2015-06-22 at 14:29 +0100, Liam Girdwood wrote:
> > > > On Mon, 2015-06-22 at 15:23 +0200, Takashi Iwai wrote:
> > > > > At Mon, 22 Jun 2015 14:54:29 +0200,
> > > > > Daniel Vetter wrote:
> > > > > > 
> > > > > > On Fri, Jun 19, 2015 at 01:15:57PM +0200, Takashi Iwai wrote:
> > > > > > > At Fri, 19 Jun 2015 20:33:39 +1000,
> > > > > > > Dave Airlie wrote:
> > > > > > > > 
> > > > > > > > On 19 June 2015 at 19:54, Lin, Mengdong <mengdong.lin at intel.com> wrote:
> > > > > > > > > Hi Takashi/Dave,
> > > > > > > > >
> > > > > > > > > Shall we move or cc this discussion on audio driver side to ALSA ML?
> > > > > > > > 
> > > > > > > > Oops I thought I had cc'ed these patches to alsa-devel as well when I sent them.
> > > > > > > > 
> > > > > > > > > I think we also need to decide how to manage PCM devices for DP MST.
> > > > > > > > > Now the HD-A driver create a PCM device for each pin, and the substream
> > > > > > > > > number is 1 for each PCM. Now with DP MST enabled, each pin can support
> > > > > > > > > multiple streams (e.g. 3 on Intel HSW/BDW/SKL).
> > > > > > > > >
> > > > > > > > > There may be 2 options:
> > > > > > > > > -#1: Let an HDMI codec specify number of substreams, same as the number
> > > > > > > > > of device entries on a pin. We can specify 3 for HSW/BDW/SKL. Other
> > > > > > > > > vendors can also specify a value according to actual HW capabilities.
> > > > > > > > >
> > > > > > > > > So for HSW, we have 3x3 subtreams totally. But we only have 3 convertors
> > > > > > > > > (for 3 display pipelines), so we can open up to 3 substreams at the same
> > > > > > > > > time. When the audio driver finds all 3 convertors are used when opening
> > > > > > > > > a new substream, it will fail.
> > > > > > > > 
> > > > > > > > One thing I noticed is the number of devices on a PIN is only updated when
> > > > > > > > the MST device is plugged in so normally pins 5,6,7 have 0 devices, and when
> > > > > > > > I plug in MST device, I get the 3 devices on port 6. So it seems dynamic
> > > > > > > > enough at this point, though I guess it'll always be 0 or 3.
> > > > > > > > >
> > > > > > > > > - #2: Create PCM device dynamically. Only create a PCM devices for a device
> > > > > > > > > entry which connects to monitor with audio support. When the monitor
> > > > > > > > > is removed, the PCM device will be disconnected, closed and removed,
> > > > > > > > > similar to the USB case.
> > > > > > > > >
> > > > > > > > > This will change ALSA core. But there will be less PCM devices and
> > > > > > > > > substreams, since the number of connected monitors Is decided by the
> > > > > > > > > actual GPU display pipeline.
> > > > > > > > 
> > > > > > > > I like this option more, since I think it should be more like USB, but I've
> > > > > > > > no idea how much work it would be from the alsa side, this patch was
> > > > > > > > probably as deep into alsa as I've gone.
> > > > > > > 
> > > > > > > Two things have to be considered for compatibility:
> > > > > > > - ELD, channel map and jack detection: these are created per PCM
> > > > > > >   device, and extending to substream would confuse user space a lot.
> > > > > > >   In theory, it can be extended using subdevice number, but in anyway
> > > > > > >   this won't work with PulseAudio as is.
> > > > > > > 
> > > > > > > - The per-pin assignment provides a more or less persistent route to a
> > > > > > >   certain device.  Changing the assignment method may break the
> > > > > > >   previous setup.
> > > > > > > 
> > > > > > > Also, the dynamic PCM creation / removal is an issue that has been
> > > > > > > discussed many times.  Unfortunately it won't work as is, at lest for
> > > > > > > PA.  Currently PA does probing of devices only at the card probe time.
> > > > > > > The hotplug of USB-audio works because it's always tied with the
> > > > > > > card.  But in this case, the card remains while only the PCM devices
> > > > > > > will be created / removed, thus the probe in PA won't be triggered.
> > > > > > 
> > > > > > I guess that means we either have to hotplug entire (fake) cards or fix up
> > > > > > userspace to support mst audio properly?
> > > > > 
> > > > > It would work for HSW/BDW.  But BYT/BSW and SKL share the same
> > > > > controller for both HDMI and analog codecs, thus the card can't be
> > > > > handled as hotplugged.
> > > > > 
> > > > > > We had to do some minimal changes
> > > > > > to the ddx drivers too to make sure they're rescanning the connector list
> > > > > > properly. Imo since this is all new I think we could require PA to rescan
> > > > > > the PCM dev list on hotplug events too to support DP MST. And we kinda do
> > > > > > need hotplug at that level since if we'd hotplug the entire card we'd kill
> > > > > > a stream that's running on some other display.
> > > > > > 
> > > > > > And always registering all of them feels like a very bad hack too.
> > > > > 
> > > > > Yeah, I personally am for the PCM hotplug, too.  It's a cleaner way.
> > > > > 
> > > > > (The current static assignment comes from the chips where they had no
> > > > >  ELD communication -- the hardware before the recent onboard Intel and
> > > > >  AMD gfx but connected somehow externally.  For such hardware, we
> > > > >  still need the static assignment.)
> > > > > 
> > > > > OTOH, we have to keep some compatibility.  And moving to the hotplug
> > > > > would break certainly some existing configuration that assumes the
> > > > > static port / device assignment.
> > > > > 
> > > > > So, a compromise would be:
> > > > > - change the behavior via either Kconfig or module option.
> > > > > - create many PCM devices statically as much as possible
> > > > > 
> > > > > The latter can be a problem in the case of AMD or Nvidia where 8 (or
> > > > > more?) ports may exist.  The former is, of course, messy and confusing
> > > > > for users, too.
> > > > 
> > > > Fwiw, Mengdong has some patches that are work in progress for the
> > > > dynamic PCM creation/removal as part of the topology work. The topology
> > > > has to support DSP FW that can load/unload different services that may
> > > > also include a dynamically registered PCM/compressed PCM device.
> > > > 
> > > > Tanu, what's your take on the effort for dynamically created PCMs
> > > > support for pulseaudio ?
> > > 
> > > It's a significant amount of work, but I think PulseAudio should be
> > > improved to support this in any case, if other approaches make life
> > > miserable for driver developers.
> > > 
> > > What would be the interface for getting notifications about new and
> > > removed PCM devices? udev?
> > 
> > In general, yes.
> > 
> > > PulseAudio (mostly) doesn't use the hw:X devices directly. Instead, it
> > > uses logical names like "front", "hdmi", "iec958", etc. Speaking of HDMI
> > > specifically, PulseAudio uses devices from "hdmi:X,0" to "hdmi:X,7".
> > > With this new dynamic PCM system, do these logical names still work?
> > 
> > Unfortunately, this doesn't work for HDA as is, because of the
> > terribly arcane secret.  For keeping the compatibility with the old
> > config, there is a static mapping of the hdmi:x,y and hw:x,z.
> > 
> > Maybe we should introduce a new device class for dynamic HDMI/DP
> > device, something like dhdmi:x,y, to make things straightforward.
> > (Just a concept -- I'm not good at naming.)
> > 
> > Alternatively, we may introduce a new argument to hdmi PCM to access
> > like "hdmi:CARD=x,SYSDEV=y".
> 
> What happens if PulseAudio tries to open "dhdmi:x,y" or
> "hdmi:CARD=x,SYSDEV=y", when y points to a non-HDMI device? If it fails,
> then PulseAudio can replace its current "hdmi:x,[0-7]" devices with
> "hdmi:CARD=X,SYSDEV=[0-13]" and blindly try every hw PCM device. But if
> opening a non-HDMI device through "hdmi:CARD=x,SYSDEV=y" succeeds, then
> how is PulseAudio supposed to know which hw PCM devices are HDMI
> devices?

It's a good question.  I think this is the core part of the missing
pieces.

What I have in my mind is to extend SNDRV_PCM_CLASS_* definition for
dedicated to HDMI/DP, e.g. SNDRV_PCM_CLASS_HDMI.  The difference
between DP and HDMI can be specified in subclass optionally.
We may add also SNDRV_PCM_CLASS_SPDIF, too.

This can be taken from snd_pcm_info_get_class().  Also it's exposed in
sysfs, too.  (Oh the sysfs interface looks buggy, I have to fix...)


Takashi