[pulseaudio-discuss] Bluetooth codec selection

Tue Mar 17 09:23:22 UTC 2020

On Sun, 2020-01-12 at 12:42 +0100, Pali Rohár wrote:
> On Sunday 12 January 2020 09:21:31 Tanu Kaskinen wrote:
> > On Tue, 2020-01-07 at 19:37 +0100, Pali Rohár wrote:
> > > Hello!
> > > 
> > > My A2DP patch series which adds support for more A2DP codecs is still in
> > > review state, but Tanu about year ago wrote that do not like my proposal
> > > with usage of PA profiles for selecting A2DP codec. Because nobody for
> > > last year come up with another option how to solve this problem and
> > > because without it is not possible to add support for other codecs it
> > > means that pulseaudio is stucked and has no way to improve A2DP support.
> > > 
> > > So here I'm adding another proposal how to handle multi-codec support
> > > for bluetooth in pulseaudio. But first description of our problem:
> > > 
> > > Currently we have following profiles for bluetooth card:
> > > 
> > > * headset_audio_gateway
> > > * headset_head_unit
> > > * a2dp_source
> > > * a2dp_sink
> > > 
> > > If we look at HW level of HSP/HFP profiles they may supports following
> > > codecs: CVSD, mSBC, AuriStream. All are bidirectional, synchronous.
> > > Moreover encoding from PCM stream to codec data may be done at HW level
> > > (bluetooth adapter) or on SW level (pulseaudio). Currently encoding to
> > > CVSD is done at HW level and encoding to mSBC at SW level. But e.g. new
> > > Thinkpads has bluetooth adapters which can do mSBC-encoding at HW level
> > > and theoretically pulseaudio could just pass PCM samples to BT without
> > > need to use any software sbc encoding library (*).
> > > 
> > > And if we look at A2DP profiles it is quite a bit complicated. Every
> > > "A2DP codec" is either one-way (source or sink) or is bi-directional
> > > (but it is rare and most devices does not support them). Plus every
> > > "A2DP codec" may support more "codec profiles", so e.g. we have SBC-MQ
> > > or SBC-HQ. And bidirectional "A2DP codecs" may use different "audio
> > > codec" for one directional and different for back directional (e.g.
> > > A2DP codec aptX-LL uses aptX codec for playing and SBC codec for
> > > recording).
> > > 
> > > Result is: "SBC" codec is supported in "headset_head_unit" PA profile
> > > (as mSBC"), it is supported also in "a2dp_sink" PA profile when "A2DP
> > > SBC" codec is used and also in "a2dp_source" PA profile when "aptX-LL"
> > > codec is used.
> > > 
> > > So adding a new API which sets codec on PA bluetooth card is not enough.
> > > 
> > > I would propose following API:
> > > 
> > > Add support for pulseaudio "sub-profiles". For every pulseaudio profile
> > > it would be possible to register sub-profile and applications via
> > > pulseaudio API would be able to change sub-profile of some PA card (if
> > > currently selected PA profile would support it).
> > > 
> > > Plus adds a two new profiles:
> > > * a2dp_source_with_backchannel
> > > * a2dp_sink_with_backchannel
> > > 
> > > For music playback in most cases is useful only profiles without
> > > backchannel to maximize bandwidth. And because A2DP codecs with
> > > bachannel does not fit into HSP profiles it make sense to have them in
> > > separate profile. In most cases A2DP profile with backchannel is the
> > > best way for VOIP (as it has microphone recording support), so bluetooth
> > > policy plugin would be extended to use also this profile for VOIP.
> > > 
> > > Sub-profiles for bluetooth PA card would be following:
> > > 
> > > * headset_audio_gateway
> > >   - CVSD (hardware)
> > >   - mSBC (software)
> > >   - mSBC (hardware)
> > >   ...
> > > * headset_head_unit
> > >   - same as in headset_audio_gateway
> > > * a2dp_source
> > >   - SBC-LQ
> > >   - SBC-MQ
> > >   - SBC-HQ
> > >   ..,
> > >   - faststream without backchannel
> > >   - aptX
> > >   - aptX-HD
> > >   - aptX-LL without backannel
> > >   - LDAC
> > >   ...
> > > * a2dp_sink
> > >   - same as in a2dp_source
> > > * a2dp_source_with_backchannel
> > >   - faststream with backchannel
> > >   - aptX-LL with backchannel
> > > * a2dp_sink_with_backchannel
> > >   - same as in a2dp_source_with_backchannel
> > > 
> > > So sub-profile would be one specific configuration of codec. In HSP/HFP
> > > there would be two sub-profiles for every codec (one for software
> > > encoding, one for hardware encoding). In A2DP it may be various. E.g.
> > > for "SBC profiles" there would be one sub-profile for every SBC
> > > configuration. For faststream codec there would be two, one with
> > > backchannel, one without. Default sub-codec selection would be up to
> > > the pulseaudio module. And applications (pactl / pavucontrol / ...)
> > > would be able to change PA sub-profile in similar way how they can do it
> > > for PA profiles.
> > > 
> > > Of course some sub-profiles does not have to be supported (e.g. mSBC in
> > > hardware encoding) and in these cases pulseaudio would not announce
> > > these unsupported sub-profiles.
> > > 
> > > What do you think about this my proposal?
> > > 
> > > Tanu, is this acceptable for you?
> > > 
> > > Basically it is needed only to add APIs for applications to:
> > > * get current sub-profile of card
> > > * get list of all sub-profiles for specific profile
> > > * change sub-profile of card
> > > 
> > > I'm not saying how would API look like. I'm open for implementation
> > > details... I can imagine that we can extend PA protocol or use messaging
> > > API for it or anything else...
> > > 
> > > 
> > > (*) - currently Linux kernel blocks this usage of mSBC HW encoding and
> > > force userspace to do whole encoding at application level (in
> > > pulseaudio).
> > 
> > Thanks for the proposal, I read it now and all the different use cases
> > seem to fit quite neatly to the sub-profile model, so it's a good
> > starting point at least.
> > 
> > I'm trying to think this from the user's point of view. The proposed UI
> > seems to be that first the sub-profile is chosen (activated) and then
> > the codec parameters can be chosen.
> 
> On Sunday 12 January 2020 09:23:52 Tanu Kaskinen wrote:
> > I meant "first the profile is chosen", not sub-profile.
> 
> Yes, first user chose profile and then sub-profile (= codec parameters).
> 
> > I think it would be better to have
> > all codec options visible regardless of the currently active profile.
> > Some profiles aren't meant to be manually chosen (a2dp_source,
> > headset_audio_gateway)
> 
> Bluetooth may support both a2dp_sink and a2dp_source profiles. And it
> may support different set of A2DP codecs on these profiles. (E.g. it can
> encode LDAC, but cannot decode LDAC).
> 
> So we cannot make visible all options as some of them would not make
> sense.
> 
> We can e.g. put all codecs into "sub-profile" select box, but codecs not
> "compatible" with currently selected profile would be gray-out and user
> would not be able to choose it. But he would see that PA support this
> option in different configuration. Or add some other GUI where user
> choose codec (=sub-profile) and then another select-box would find all
> compatible profiles for this codec, so he could choose what he want.
> 
> But this is just designing UI for selecting sub-profile. It does not
> involve any changes on pulseaudio server.
> 
> > and it's probably good to avoid unnecessary
> > manual changes between a2dp_sink and headset_head_unit too, because
> > that may confuse or make things more complicated for the automatic
> > profile switching logic.
> 
> We cannot avoid it. Choosing codec is part of connection establishment.
> So if user is not satisfied with currently used codec he should be able
> to change it.
> 
> > Setting the codec preferences and choosing the
> > active profile would be good to keep as separate actions.
> > 
> > Are there differences in use cases for HFP and bidirectional A2DP? To
> 
> I guess you are asking for "bidirectional A2DP codec with activated
> backchannel". As I saw, aptX-LL is often used without backchannel, so is
> only one-directional.
> 
> And what are differences? HFP is synchronous with a low delay and
> supports additional features, e.g. phone control, battery power status,
> button press, vendor extensions, etc...
> 
> Bidirectional A2DP codec FastStream has better quality then HFP CVSD.
> Some devices does not have to support HFP mSBC.
> 
> So user had to choose: Does it need better quality? Then choose A2DP.
> Does it need to control phone calls? Then choose HFP with lower audio
> quality.
> 
> I'm looking at it: You need to choose profile (HFP or A2DP) based on
> what features you need. And then choose audio quality.
> 
> All bluetooth audio devices describes which bluetooth profiles (HFP,
> A2DP, ...) they support and also what functionality they in their
> profile they support. So user needs to know what is he using to not be
> surprised that something does not work (just because PA decided to use
> other bluetooth profile).
> 
> > me it seems that the use cases are the same, and the user should have
> > the HFP and bidirectional A2DP codecs in one list (not necessarily in
> > this order):
> > 
> > * CVSD (hardware)
> > * mSBC (software)
> > * mSBC (hardware)
> > * AuriStream
> > * FastStream
> > * aptX-LL
> > 
> > So I'm not sure the new bidirectional A2DP profiles are needed, but
> > maybe having them makes the implementation easier. I'd be ok with
> > adding the new profiles, but making the primary codec configuration API
> > separate and automatically switching the profile when the preferred
> > bidirectional codec is changed.
> > 
> > I noticed you listed "aptX-LL without backchannel" as a sub-profile.
> > Isn't that the same as plain aptX?
> 
> No. Those are two different things. aptX-LL should have lower latency.
> Encoded stream is same, but packed differently at lower bluetooth
> layers (different buffer sizes and maybe on receiver side also some
> optimizations).
> 
> > Would you be willing to document the codec negotiation process in the
> > wiki?
> 
> What exactly?

The things that you mentioned below:
 * Either side can establish the connection, so it's not obvious what
things can trigger a connection. For example, what might cause a
headset to initiate the connection?
 * BlueZ may decide the codec instead of PulseAudio. What logic does it
use?
 * There are many places with their own defaults. What are these places
and their codec selection logic?

Maybe these things doesn't affect the UI as I first thought, but it
would be great to have a reference anyway, so that the system can be
understood when debugging codec issues.

> > I can give you a wiki account. I believe you have already
> > explained this before in emails, but I've forgotten the details. To me
> > it would make sense if only the playback side (or the audio gateway for
> > bidirectional modes) could decide the codec, but I recall the reality
> > is more complicated, and there were multiple places where the chosen
> > codec may be remembered.
> 
> Side which establishing connection choose codec. It does not have to be
> playback side. And connection establishment is done by calling DBus
> bluez function, independently of pulseaudio (in most cases desktop
> bluetooth GUI application do that). So we cannot avoid it.
> 
> But there are many places which has own defaults. So establishing
> connection would work without specifying codec and some remembered /
> default value would be used.
> 
> > The negotiation details affect the constraints
> > for UI design.
> 
> We can do this: When user choose profile, then PA issue command to
> activate profile with some default codec. User would see in GUI what was
> chosen and could change if is not happy with it. If he does not care
> about it he would use default / remembered option and so does not have
> to choose anything.

Here's my current overall thinking:

Having the six profiles as you suggested is good.

If I understood correctly, you're proposing to make the sub-profiles a
generic concept rather than a bluetooth specific API. I'm a bit
hesitant to add an abstraction that doesn't in reality abstract
anything, because bluetooth is the only user for now and other users
haven't been suggested. On the other hand, I don't foresee concrete
problems with this either, so I guess it doesn't really matter.

In my previous mail I suggested keeping the codec preference
configuration independent from the active profile, but this can be
implemented later if this is something people want.

-- 
Tanu

https://www.patreon.com/tanuk
https://liberapay.com/tanuk