[Intel-gfx] [PATCH 3/4] snd: add support for displayport multi-stream to hda codec.

[Takashi Iwai]

At Mon, 22 Jun 2015 15:21:16 +0000,
Kaskinen, Tanu wrote:
> 
> On Mon, 2015-06-22 at 14:29 +0100, Liam Girdwood wrote:
> > On Mon, 2015-06-22 at 15:23 +0200, Takashi Iwai wrote:
> > > At Mon, 22 Jun 2015 14:54:29 +0200,
> > > Daniel Vetter wrote:
> > > > 
> > > > On Fri, Jun 19, 2015 at 01:15:57PM +0200, Takashi Iwai wrote:
> > > > > At Fri, 19 Jun 2015 20:33:39 +1000,
> > > > > Dave Airlie wrote:
> > > > > > 
> > > > > > On 19 June 2015 at 19:54, Lin, Mengdong <mengdong.lin at intel.com> wrote:
> > > > > > > Hi Takashi/Dave,
> > > > > > >
> > > > > > > Shall we move or cc this discussion on audio driver side to ALSA ML?
> > > > > > 
> > > > > > Oops I thought I had cc'ed these patches to alsa-devel as well when I sent them.
> > > > > > 
> > > > > > > I think we also need to decide how to manage PCM devices for DP MST.
> > > > > > > Now the HD-A driver create a PCM device for each pin, and the substream
> > > > > > > number is 1 for each PCM. Now with DP MST enabled, each pin can support
> > > > > > > multiple streams (e.g. 3 on Intel HSW/BDW/SKL).
> > > > > > >
> > > > > > > There may be 2 options:
> > > > > > > -#1: Let an HDMI codec specify number of substreams, same as the number
> > > > > > > of device entries on a pin. We can specify 3 for HSW/BDW/SKL. Other
> > > > > > > vendors can also specify a value according to actual HW capabilities.
> > > > > > >
> > > > > > > So for HSW, we have 3x3 subtreams totally. But we only have 3 convertors
> > > > > > > (for 3 display pipelines), so we can open up to 3 substreams at the same
> > > > > > > time. When the audio driver finds all 3 convertors are used when opening
> > > > > > > a new substream, it will fail.
> > > > > > 
> > > > > > One thing I noticed is the number of devices on a PIN is only updated when
> > > > > > the MST device is plugged in so normally pins 5,6,7 have 0 devices, and when
> > > > > > I plug in MST device, I get the 3 devices on port 6. So it seems dynamic
> > > > > > enough at this point, though I guess it'll always be 0 or 3.
> > > > > > >
> > > > > > > - #2: Create PCM device dynamically. Only create a PCM devices for a device
> > > > > > > entry which connects to monitor with audio support. When the monitor
> > > > > > > is removed, the PCM device will be disconnected, closed and removed,
> > > > > > > similar to the USB case.
> > > > > > >
> > > > > > > This will change ALSA core. But there will be less PCM devices and
> > > > > > > substreams, since the number of connected monitors Is decided by the
> > > > > > > actual GPU display pipeline.
> > > > > > 
> > > > > > I like this option more, since I think it should be more like USB, but I've
> > > > > > no idea how much work it would be from the alsa side, this patch was
> > > > > > probably as deep into alsa as I've gone.
> > > > > 
> > > > > Two things have to be considered for compatibility:
> > > > > - ELD, channel map and jack detection: these are created per PCM
> > > > >   device, and extending to substream would confuse user space a lot.
> > > > >   In theory, it can be extended using subdevice number, but in anyway
> > > > >   this won't work with PulseAudio as is.
> > > > > 
> > > > > - The per-pin assignment provides a more or less persistent route to a
> > > > >   certain device.  Changing the assignment method may break the
> > > > >   previous setup.
> > > > > 
> > > > > Also, the dynamic PCM creation / removal is an issue that has been
> > > > > discussed many times.  Unfortunately it won't work as is, at lest for
> > > > > PA.  Currently PA does probing of devices only at the card probe time.
> > > > > The hotplug of USB-audio works because it's always tied with the
> > > > > card.  But in this case, the card remains while only the PCM devices
> > > > > will be created / removed, thus the probe in PA won't be triggered.
> > > > 
> > > > I guess that means we either have to hotplug entire (fake) cards or fix up
> > > > userspace to support mst audio properly?
> > > 
> > > It would work for HSW/BDW.  But BYT/BSW and SKL share the same
> > > controller for both HDMI and analog codecs, thus the card can't be
> > > handled as hotplugged.
> > > 
> > > > We had to do some minimal changes
> > > > to the ddx drivers too to make sure they're rescanning the connector list
> > > > properly. Imo since this is all new I think we could require PA to rescan
> > > > the PCM dev list on hotplug events too to support DP MST. And we kinda do
> > > > need hotplug at that level since if we'd hotplug the entire card we'd kill
> > > > a stream that's running on some other display.
> > > > 
> > > > And always registering all of them feels like a very bad hack too.
> > > 
> > > Yeah, I personally am for the PCM hotplug, too.  It's a cleaner way.
> > > 
> > > (The current static assignment comes from the chips where they had no
> > >  ELD communication -- the hardware before the recent onboard Intel and
> > >  AMD gfx but connected somehow externally.  For such hardware, we
> > >  still need the static assignment.)
> > > 
> > > OTOH, we have to keep some compatibility.  And moving to the hotplug
> > > would break certainly some existing configuration that assumes the
> > > static port / device assignment.
> > > 
> > > So, a compromise would be:
> > > - change the behavior via either Kconfig or module option.
> > > - create many PCM devices statically as much as possible
> > > 
> > > The latter can be a problem in the case of AMD or Nvidia where 8 (or
> > > more?) ports may exist.  The former is, of course, messy and confusing
> > > for users, too.
> > 
> > Fwiw, Mengdong has some patches that are work in progress for the
> > dynamic PCM creation/removal as part of the topology work. The topology
> > has to support DSP FW that can load/unload different services that may
> > also include a dynamically registered PCM/compressed PCM device.
> > 
> > Tanu, what's your take on the effort for dynamically created PCMs
> > support for pulseaudio ?
> 
> It's a significant amount of work, but I think PulseAudio should be
> improved to support this in any case, if other approaches make life
> miserable for driver developers.
> 
> What would be the interface for getting notifications about new and
> removed PCM devices? udev?

In general, yes.

> PulseAudio (mostly) doesn't use the hw:X devices directly. Instead, it
> uses logical names like "front", "hdmi", "iec958", etc. Speaking of HDMI
> specifically, PulseAudio uses devices from "hdmi:X,0" to "hdmi:X,7".
> With this new dynamic PCM system, do these logical names still work?

Unfortunately, this doesn't work for HDA as is, because of the
terribly arcane secret.  For keeping the compatibility with the old
config, there is a static mapping of the hdmi:x,y and hw:x,z.

Maybe we should introduce a new device class for dynamic HDMI/DP
device, something like dhdmi:x,y, to make things straightforward.
(Just a concept -- I'm not good at naming.)

Alternatively, we may introduce a new argument to hdmi PCM to access
like "hdmi:CARD=x,SYSDEV=y".

> When probing a new card, PulseAudio goes through the configured set of
> profiles for that card, and tries to open each configured device
> separately, and also each configured combination of devices (one profile
> may consist of multiple devices). This kind of probing gets messy, if
> devices appear dynamically when the card is already in use, because
> reliable results require other devices of the card to be closed before
> trying to open the new device. To avoid glitches in playing audio, the
> probing would have to be delayed until the card is idle... I suspect
> that isn't really feasible, so maybe we will just have to live with
> glitches.
> 
> It would be nice if PulseAudio didn't have to open the PCM devices when
> probing, but that would require some interface to find out which PCM
> devices can be opened simultaneously and which can not.

Yeah, that's the missing information.  We can give something in
alsa-lib config to indicate the conflict, in theory...

> Another problem is that PulseAudio doesn't know which hw PCM devices are
> used by the logical devices. Without that information, PulseAudio has to
> re-probe every device that failed previously, when new hw PCM devices
> appear, and also re-probe every device that succeeded previously, when
> PCM devices are removed. I suspect that this information might actually
> be available, since "aplay -v -Dfront" shows the underlying hw device.
> Maybe PulseAudio should be using that information?

aplay just uses an API function to dump the setup, and its text is not 
supposed to be parsed as an interface.  We need a proper API function
to provide that missing piece like above.


> > Btw, the topology core now also dynamically
> > creates/removes mixer controls, can PA handle this atm ?
> 
> No, PA checks the mixer controls only when loading a new card.
> Dynamically added controls are ignored. Dynamically removed controls
> just cause silent failure, at least when setting volume (I didn't check
> other use cases). That is, changing the volume appears to succeed, but
> nothing actually happens.

Won't PA use ELD or other information?  The corresponding controls to
HDMI/DP will be created / deleted dynamically together with a PCM
device, I suppose.


Takashi