[pulseaudio-discuss] Bluetooth codec selection

Sun Jan 12 11:42:00 UTC 2020

On Sunday 12 January 2020 09:21:31 Tanu Kaskinen wrote:
> On Tue, 2020-01-07 at 19:37 +0100, Pali Rohár wrote:
> > Hello!
> > 
> > My A2DP patch series which adds support for more A2DP codecs is still in
> > review state, but Tanu about year ago wrote that do not like my proposal
> > with usage of PA profiles for selecting A2DP codec. Because nobody for
> > last year come up with another option how to solve this problem and
> > because without it is not possible to add support for other codecs it
> > means that pulseaudio is stucked and has no way to improve A2DP support.
> > 
> > So here I'm adding another proposal how to handle multi-codec support
> > for bluetooth in pulseaudio. But first description of our problem:
> > 
> > Currently we have following profiles for bluetooth card:
> > 
> > * headset_audio_gateway
> > * headset_head_unit
> > * a2dp_source
> > * a2dp_sink
> > 
> > If we look at HW level of HSP/HFP profiles they may supports following
> > codecs: CVSD, mSBC, AuriStream. All are bidirectional, synchronous.
> > Moreover encoding from PCM stream to codec data may be done at HW level
> > (bluetooth adapter) or on SW level (pulseaudio). Currently encoding to
> > CVSD is done at HW level and encoding to mSBC at SW level. But e.g. new
> > Thinkpads has bluetooth adapters which can do mSBC-encoding at HW level
> > and theoretically pulseaudio could just pass PCM samples to BT without
> > need to use any software sbc encoding library (*).
> > 
> > And if we look at A2DP profiles it is quite a bit complicated. Every
> > "A2DP codec" is either one-way (source or sink) or is bi-directional
> > (but it is rare and most devices does not support them). Plus every
> > "A2DP codec" may support more "codec profiles", so e.g. we have SBC-MQ
> > or SBC-HQ. And bidirectional "A2DP codecs" may use different "audio
> > codec" for one directional and different for back directional (e.g.
> > A2DP codec aptX-LL uses aptX codec for playing and SBC codec for
> > recording).
> > 
> > Result is: "SBC" codec is supported in "headset_head_unit" PA profile
> > (as mSBC"), it is supported also in "a2dp_sink" PA profile when "A2DP
> > SBC" codec is used and also in "a2dp_source" PA profile when "aptX-LL"
> > codec is used.
> > 
> > So adding a new API which sets codec on PA bluetooth card is not enough.
> > 
> > I would propose following API:
> > 
> > Add support for pulseaudio "sub-profiles". For every pulseaudio profile
> > it would be possible to register sub-profile and applications via
> > pulseaudio API would be able to change sub-profile of some PA card (if
> > currently selected PA profile would support it).
> > 
> > Plus adds a two new profiles:
> > * a2dp_source_with_backchannel
> > * a2dp_sink_with_backchannel
> > 
> > For music playback in most cases is useful only profiles without
> > backchannel to maximize bandwidth. And because A2DP codecs with
> > bachannel does not fit into HSP profiles it make sense to have them in
> > separate profile. In most cases A2DP profile with backchannel is the
> > best way for VOIP (as it has microphone recording support), so bluetooth
> > policy plugin would be extended to use also this profile for VOIP.
> > 
> > Sub-profiles for bluetooth PA card would be following:
> > 
> > * headset_audio_gateway
> >   - CVSD (hardware)
> >   - mSBC (software)
> >   - mSBC (hardware)
> >   ...
> > * headset_head_unit
> >   - same as in headset_audio_gateway
> > * a2dp_source
> >   - SBC-LQ
> >   - SBC-MQ
> >   - SBC-HQ
> >   ..,
> >   - faststream without backchannel
> >   - aptX
> >   - aptX-HD
> >   - aptX-LL without backannel
> >   - LDAC
> >   ...
> > * a2dp_sink
> >   - same as in a2dp_source
> > * a2dp_source_with_backchannel
> >   - faststream with backchannel
> >   - aptX-LL with backchannel
> > * a2dp_sink_with_backchannel
> >   - same as in a2dp_source_with_backchannel
> > 
> > So sub-profile would be one specific configuration of codec. In HSP/HFP
> > there would be two sub-profiles for every codec (one for software
> > encoding, one for hardware encoding). In A2DP it may be various. E.g.
> > for "SBC profiles" there would be one sub-profile for every SBC
> > configuration. For faststream codec there would be two, one with
> > backchannel, one without. Default sub-codec selection would be up to
> > the pulseaudio module. And applications (pactl / pavucontrol / ...)
> > would be able to change PA sub-profile in similar way how they can do it
> > for PA profiles.
> > 
> > Of course some sub-profiles does not have to be supported (e.g. mSBC in
> > hardware encoding) and in these cases pulseaudio would not announce
> > these unsupported sub-profiles.
> > 
> > What do you think about this my proposal?
> > 
> > Tanu, is this acceptable for you?
> > 
> > Basically it is needed only to add APIs for applications to:
> > * get current sub-profile of card
> > * get list of all sub-profiles for specific profile
> > * change sub-profile of card
> > 
> > I'm not saying how would API look like. I'm open for implementation
> > details... I can imagine that we can extend PA protocol or use messaging
> > API for it or anything else...
> > 
> > 
> > (*) - currently Linux kernel blocks this usage of mSBC HW encoding and
> > force userspace to do whole encoding at application level (in
> > pulseaudio).
> 
> Thanks for the proposal, I read it now and all the different use cases
> seem to fit quite neatly to the sub-profile model, so it's a good
> starting point at least.
> 
> I'm trying to think this from the user's point of view. The proposed UI
> seems to be that first the sub-profile is chosen (activated) and then
> the codec parameters can be chosen.

On Sunday 12 January 2020 09:23:52 Tanu Kaskinen wrote:
> I meant "first the profile is chosen", not sub-profile.

Yes, first user chose profile and then sub-profile (= codec parameters).

> I think it would be better to have
> all codec options visible regardless of the currently active profile.
> Some profiles aren't meant to be manually chosen (a2dp_source,
> headset_audio_gateway)

Bluetooth may support both a2dp_sink and a2dp_source profiles. And it
may support different set of A2DP codecs on these profiles. (E.g. it can
encode LDAC, but cannot decode LDAC).

So we cannot make visible all options as some of them would not make
sense.

We can e.g. put all codecs into "sub-profile" select box, but codecs not
"compatible" with currently selected profile would be gray-out and user
would not be able to choose it. But he would see that PA support this
option in different configuration. Or add some other GUI where user
choose codec (=sub-profile) and then another select-box would find all
compatible profiles for this codec, so he could choose what he want.

But this is just designing UI for selecting sub-profile. It does not
involve any changes on pulseaudio server.

> and it's probably good to avoid unnecessary
> manual changes between a2dp_sink and headset_head_unit too, because
> that may confuse or make things more complicated for the automatic
> profile switching logic.

We cannot avoid it. Choosing codec is part of connection establishment.
So if user is not satisfied with currently used codec he should be able
to change it.

> Setting the codec preferences and choosing the
> active profile would be good to keep as separate actions.
> 
> Are there differences in use cases for HFP and bidirectional A2DP? To

I guess you are asking for "bidirectional A2DP codec with activated
backchannel". As I saw, aptX-LL is often used without backchannel, so is
only one-directional.

And what are differences? HFP is synchronous with a low delay and
supports additional features, e.g. phone control, battery power status,
button press, vendor extensions, etc...

Bidirectional A2DP codec FastStream has better quality then HFP CVSD.
Some devices does not have to support HFP mSBC.

So user had to choose: Does it need better quality? Then choose A2DP.
Does it need to control phone calls? Then choose HFP with lower audio
quality.

I'm looking at it: You need to choose profile (HFP or A2DP) based on
what features you need. And then choose audio quality.

All bluetooth audio devices describes which bluetooth profiles (HFP,
A2DP, ...) they support and also what functionality they in their
profile they support. So user needs to know what is he using to not be
surprised that something does not work (just because PA decided to use
other bluetooth profile).

> me it seems that the use cases are the same, and the user should have
> the HFP and bidirectional A2DP codecs in one list (not necessarily in
> this order):
> 
> * CVSD (hardware)
> * mSBC (software)
> * mSBC (hardware)
> * AuriStream
> * FastStream
> * aptX-LL
> 
> So I'm not sure the new bidirectional A2DP profiles are needed, but
> maybe having them makes the implementation easier. I'd be ok with
> adding the new profiles, but making the primary codec configuration API
> separate and automatically switching the profile when the preferred
> bidirectional codec is changed.
> 
> I noticed you listed "aptX-LL without backchannel" as a sub-profile.
> Isn't that the same as plain aptX?

No. Those are two different things. aptX-LL should have lower latency.
Encoded stream is same, but packed differently at lower bluetooth
layers (different buffer sizes and maybe on receiver side also some
optimizations).

> Would you be willing to document the codec negotiation process in the
> wiki?

What exactly?

> I can give you a wiki account. I believe you have already
> explained this before in emails, but I've forgotten the details. To me
> it would make sense if only the playback side (or the audio gateway for
> bidirectional modes) could decide the codec, but I recall the reality
> is more complicated, and there were multiple places where the chosen
> codec may be remembered.

Side which establishing connection choose codec. It does not have to be
playback side. And connection establishment is done by calling DBus
bluez function, independently of pulseaudio (in most cases desktop
bluetooth GUI application do that). So we cannot avoid it.

But there are many places which has own defaults. So establishing
connection would work without specifying codec and some remembered /
default value would be used.

> The negotiation details affect the constraints
> for UI design.

We can do this: When user choose profile, then PA issue command to
activate profile with some default codec. User would see in GUI what was
chosen and could change if is not happy with it. If he does not care
about it he would use default / remembered option and so does not have
to choose anything.

-- 
Pali Rohár
pali.rohar at gmail.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/pulseaudio-discuss/attachments/20200112/0e1f82f4/attachment.sig>