[pulseaudio-discuss] [Intel-gfx] [PATCH 3/4] snd: add support for displayport multi-stream to hda codec.

Tue Jun 23 00:51:22 PDT 2015

(Added pulseaudio-discuss to CC.)

On Mon, 2015-06-22 at 17:44 +0200, Takashi Iwai wrote:
> At Mon, 22 Jun 2015 15:21:16 +0000,
> Kaskinen, Tanu wrote:
> > 
> > On Mon, 2015-06-22 at 14:29 +0100, Liam Girdwood wrote:
> > > On Mon, 2015-06-22 at 15:23 +0200, Takashi Iwai wrote:
> > > > At Mon, 22 Jun 2015 14:54:29 +0200,
> > > > Daniel Vetter wrote:
> > > > > 
> > > > > On Fri, Jun 19, 2015 at 01:15:57PM +0200, Takashi Iwai wrote:
> > > > > > At Fri, 19 Jun 2015 20:33:39 +1000,
> > > > > > Dave Airlie wrote:
> > > > > > > 
> > > > > > > On 19 June 2015 at 19:54, Lin, Mengdong <mengdong.lin at intel.com> wrote:
> > > > > > > > Hi Takashi/Dave,
> > > > > > > >
> > > > > > > > Shall we move or cc this discussion on audio driver side to ALSA ML?
> > > > > > > 
> > > > > > > Oops I thought I had cc'ed these patches to alsa-devel as well when I sent them.
> > > > > > > 
> > > > > > > > I think we also need to decide how to manage PCM devices for DP MST.
> > > > > > > > Now the HD-A driver create a PCM device for each pin, and the substream
> > > > > > > > number is 1 for each PCM. Now with DP MST enabled, each pin can support
> > > > > > > > multiple streams (e.g. 3 on Intel HSW/BDW/SKL).
> > > > > > > >
> > > > > > > > There may be 2 options:
> > > > > > > > -#1: Let an HDMI codec specify number of substreams, same as the number
> > > > > > > > of device entries on a pin. We can specify 3 for HSW/BDW/SKL. Other
> > > > > > > > vendors can also specify a value according to actual HW capabilities.
> > > > > > > >
> > > > > > > > So for HSW, we have 3x3 subtreams totally. But we only have 3 convertors
> > > > > > > > (for 3 display pipelines), so we can open up to 3 substreams at the same
> > > > > > > > time. When the audio driver finds all 3 convertors are used when opening
> > > > > > > > a new substream, it will fail.
> > > > > > > 
> > > > > > > One thing I noticed is the number of devices on a PIN is only updated when
> > > > > > > the MST device is plugged in so normally pins 5,6,7 have 0 devices, and when
> > > > > > > I plug in MST device, I get the 3 devices on port 6. So it seems dynamic
> > > > > > > enough at this point, though I guess it'll always be 0 or 3.
> > > > > > > >
> > > > > > > > - #2: Create PCM device dynamically. Only create a PCM devices for a device
> > > > > > > > entry which connects to monitor with audio support. When the monitor
> > > > > > > > is removed, the PCM device will be disconnected, closed and removed,
> > > > > > > > similar to the USB case.
> > > > > > > >
> > > > > > > > This will change ALSA core. But there will be less PCM devices and
> > > > > > > > substreams, since the number of connected monitors Is decided by the
> > > > > > > > actual GPU display pipeline.
> > > > > > > 
> > > > > > > I like this option more, since I think it should be more like USB, but I've
> > > > > > > no idea how much work it would be from the alsa side, this patch was
> > > > > > > probably as deep into alsa as I've gone.
> > > > > > 
> > > > > > Two things have to be considered for compatibility:
> > > > > > - ELD, channel map and jack detection: these are created per PCM
> > > > > >   device, and extending to substream would confuse user space a lot.
> > > > > >   In theory, it can be extended using subdevice number, but in anyway
> > > > > >   this won't work with PulseAudio as is.
> > > > > > 
> > > > > > - The per-pin assignment provides a more or less persistent route to a
> > > > > >   certain device.  Changing the assignment method may break the
> > > > > >   previous setup.
> > > > > > 
> > > > > > Also, the dynamic PCM creation / removal is an issue that has been
> > > > > > discussed many times.  Unfortunately it won't work as is, at lest for
> > > > > > PA.  Currently PA does probing of devices only at the card probe time.
> > > > > > The hotplug of USB-audio works because it's always tied with the
> > > > > > card.  But in this case, the card remains while only the PCM devices
> > > > > > will be created / removed, thus the probe in PA won't be triggered.
> > > > > 
> > > > > I guess that means we either have to hotplug entire (fake) cards or fix up
> > > > > userspace to support mst audio properly?
> > > > 
> > > > It would work for HSW/BDW.  But BYT/BSW and SKL share the same
> > > > controller for both HDMI and analog codecs, thus the card can't be
> > > > handled as hotplugged.
> > > > 
> > > > > We had to do some minimal changes
> > > > > to the ddx drivers too to make sure they're rescanning the connector list
> > > > > properly. Imo since this is all new I think we could require PA to rescan
> > > > > the PCM dev list on hotplug events too to support DP MST. And we kinda do
> > > > > need hotplug at that level since if we'd hotplug the entire card we'd kill
> > > > > a stream that's running on some other display.
> > > > > 
> > > > > And always registering all of them feels like a very bad hack too.
> > > > 
> > > > Yeah, I personally am for the PCM hotplug, too.  It's a cleaner way.
> > > > 
> > > > (The current static assignment comes from the chips where they had no
> > > >  ELD communication -- the hardware before the recent onboard Intel and
> > > >  AMD gfx but connected somehow externally.  For such hardware, we
> > > >  still need the static assignment.)
> > > > 
> > > > OTOH, we have to keep some compatibility.  And moving to the hotplug
> > > > would break certainly some existing configuration that assumes the
> > > > static port / device assignment.
> > > > 
> > > > So, a compromise would be:
> > > > - change the behavior via either Kconfig or module option.
> > > > - create many PCM devices statically as much as possible
> > > > 
> > > > The latter can be a problem in the case of AMD or Nvidia where 8 (or
> > > > more?) ports may exist.  The former is, of course, messy and confusing
> > > > for users, too.
> > > 
> > > Fwiw, Mengdong has some patches that are work in progress for the
> > > dynamic PCM creation/removal as part of the topology work. The topology
> > > has to support DSP FW that can load/unload different services that may
> > > also include a dynamically registered PCM/compressed PCM device.
> > > 
> > > Tanu, what's your take on the effort for dynamically created PCMs
> > > support for pulseaudio ?
> > 
> > It's a significant amount of work, but I think PulseAudio should be
> > improved to support this in any case, if other approaches make life
> > miserable for driver developers.
> > 
> > What would be the interface for getting notifications about new and
> > removed PCM devices? udev?
> 
> In general, yes.
> 
> > PulseAudio (mostly) doesn't use the hw:X devices directly. Instead, it
> > uses logical names like "front", "hdmi", "iec958", etc. Speaking of HDMI
> > specifically, PulseAudio uses devices from "hdmi:X,0" to "hdmi:X,7".
> > With this new dynamic PCM system, do these logical names still work?
> 
> Unfortunately, this doesn't work for HDA as is, because of the
> terribly arcane secret.  For keeping the compatibility with the old
> config, there is a static mapping of the hdmi:x,y and hw:x,z.
> 
> Maybe we should introduce a new device class for dynamic HDMI/DP
> device, something like dhdmi:x,y, to make things straightforward.
> (Just a concept -- I'm not good at naming.)
> 
> Alternatively, we may introduce a new argument to hdmi PCM to access
> like "hdmi:CARD=x,SYSDEV=y".

What happens if PulseAudio tries to open "dhdmi:x,y" or
"hdmi:CARD=x,SYSDEV=y", when y points to a non-HDMI device? If it fails,
then PulseAudio can replace its current "hdmi:x,[0-7]" devices with
"hdmi:CARD=X,SYSDEV=[0-13]" and blindly try every hw PCM device. But if
opening a non-HDMI device through "hdmi:CARD=x,SYSDEV=y" succeeds, then
how is PulseAudio supposed to know which hw PCM devices are HDMI
devices?

> > When probing a new card, PulseAudio goes through the configured set of
> > profiles for that card, and tries to open each configured device
> > separately, and also each configured combination of devices (one profile
> > may consist of multiple devices). This kind of probing gets messy, if
> > devices appear dynamically when the card is already in use, because
> > reliable results require other devices of the card to be closed before
> > trying to open the new device. To avoid glitches in playing audio, the
> > probing would have to be delayed until the card is idle... I suspect
> > that isn't really feasible, so maybe we will just have to live with
> > glitches.
> > 
> > It would be nice if PulseAudio didn't have to open the PCM devices when
> > probing, but that would require some interface to find out which PCM
> > devices can be opened simultaneously and which can not.
> 
> Yeah, that's the missing information.  We can give something in
> alsa-lib config to indicate the conflict, in theory...
> 
> > Another problem is that PulseAudio doesn't know which hw PCM devices are
> > used by the logical devices. Without that information, PulseAudio has to
> > re-probe every device that failed previously, when new hw PCM devices
> > appear, and also re-probe every device that succeeded previously, when
> > PCM devices are removed. I suspect that this information might actually
> > be available, since "aplay -v -Dfront" shows the underlying hw device.
> > Maybe PulseAudio should be using that information?
> 
> aplay just uses an API function to dump the setup, and its text is not 
> supposed to be parsed as an interface.  We need a proper API function
> to provide that missing piece like above.
> 
> 
> > > Btw, the topology core now also dynamically
> > > creates/removes mixer controls, can PA handle this atm ?
> > 
> > No, PA checks the mixer controls only when loading a new card.
> > Dynamically added controls are ignored. Dynamically removed controls
> > just cause silent failure, at least when setting volume (I didn't check
> > other use cases). That is, changing the volume appears to succeed, but
> > nothing actually happens.
> 
> Won't PA use ELD or other information?  The corresponding controls to
> HDMI/DP will be created / deleted dynamically together with a PCM
> device, I suppose.

Yes, PA uses ELD. If mixer controls become dynamic too, then that's
another thing to implement.

-- 
Tanu