[pulseaudio-discuss] About extra A2DP codecs support in bluetooth module

Thu Jul 20 21:45:14 UTC 2017

Hi,

On Thu, Jul 20, 2017 at 3:42 AM, Qu Wenruo <quwenruo.btrfs at gmx.com> wrote:
>
>
> On 2017年07月20日 04:13, Luiz Augusto von Dentz wrote:
>>
>> Hi All,
>>
>> On Sat, Jul 15, 2017 at 9:08 AM, Arun Raghavan <arun at arunraghavan.net>
>> wrote:
>>>
>>>
>>>
>>> On Sat, 15 Jul 2017, at 11:37 AM, Arun Raghavan wrote:
>>>>
>>>>
>>>>
>>>> On Fri, 14 Jul 2017, at 06:50 AM, Qu Wenruo wrote:
>>>>>
>>>>>
>>>>>
>>>>> On 2017年07月12日 17:57, Arun Raghavan wrote:
>>>>>>
>>>>>> On Fri, 7 Jul 2017, at 08:35 PM, Tanu Kaskinen wrote:
>>>>>>>
>>>>>>> On Fri, 2017-07-07 at 20:33 +0800, Qu Wenruo wrote:
>>>>>>>>
>>>>>>>> After a quick glance into the code (without much knowledge about
>>>>>>>> pulseaudio), I found that pulseaudio is just using sbc library to do
>>>>>>>> the
>>>>>>>> encode.
>>>>>>>>
>>>>>>>>    From a2dp_process_render():
>>>>>>>> ---
>>>>>>>>        while (PA_LIKELY(to_encode > 0 && to_write > 0)) {
>>>>>>>>            ssize_t written;
>>>>>>>>            ssize_t encoded;
>>>>>>>>
>>>>>>>>            encoded = sbc_encode(&sbc_info->sbc,
>>>>>>>>                                 p, to_encode,
>>>>>>>>                                 d, to_write,
>>>>>>>>                                 &written);
>>>>>>>> ---
>>>>>>>>
>>>>>>>> So there is really nothing blocking us to implement other codec.
>>>>>>>> For AAC codec, just (well, without tons of preparation and setup)
>>>>>>>> call
>>>>>>>> faacEncEncode() will be the core part.
>>>>>>>> Copyright sh*t will only restrict the related library, not the PA
>>>>>>>> module.
>>>>>>>> (So if we could create a aptX codec library, then it will be
>>>>>>>> possible to
>>>>>>>> support)
>>>>>>>>
>>>>>>>> While the really hard part would be the preparation part, including
>>>>>>>> creating a structure for faac encoder to contain a faacEncHandle and
>>>>>>>> other needed info from sample rate to profile, just like sbc_info_t.
>>>>>>>>
>>>>>>>> Although I have a basic idea of what to do, I'm still figuring out
>>>>>>>> how
>>>>>>>> to handle all the details.
>>>>>>>> Like how to create an endpoint for AAC codec (codec 0 is registered
>>>>>>>> at
>>>>>>>> register_endpoint, but shouldn't it be A2DP_CODEC_SBC instead of
>>>>>>>> intermediate number 0?)
>>>>>>>
>>>>>>>
>>>>>>> I don't know bluetooth details enough to answer. I'll add Luiz to Cc
>>>>>>> in
>>>>>>> case he knows. You could try asking on the bluez mailing list too.
>>>>>>
>>>>>>
>>>>>> I would suggest just hiding away the entire RTP payloading and
>>>>>> encoding
>>>>>> using GStreamer here. That neatly sidesteps the issue of
>>>>>> hardware/software codecs, IP-sensitive codec libraries, and so forth.
>>>>>> We
>>>>>> probably want to keep the SBC path available the way it is right now
>>>>>> to
>>>>>> avoid GStreamer as a hard-dep, but otherwise, I think that's the more
>>>>>> sensible approach to this.
>>>>>
>>>>>
>>>>> I'm still fighting against dbus and bluez5 things, so GStreamer is not
>>>>> my primary goal.
>>>>> But the idea to let a framework to handle everything indeed looks neat
>>>>> and clean.
>>>>>
>>>>> However I'm more concerned about how the final A2DP profile is
>>>>> determined.
>>>>>
>>>>>   From what I can see, pulseaudio bluetooth modules just register
>>>>> endpoint for bluez, with Codec, UUID and codec specified capabilities.
>>>>> However I didn't see how codec selection is done.
>>>>>
>>>>> Is it done by bluez5 or pulseaudio?
>>>>>
>>>>> My currect understanding to implement a new codec will need:
>>>>> 1) Register a new codec in register_endpoint() of bluez5-util.c of PA.
>>>>>      Instead of codec 0 (shouldn't it be A2DP_CODEC_SBC?), we must also
>>>>> register codec
>>>>>      A2DP_CODEC_MPEG24 with a2dp_aac_t as capability.
>>>>>
>>>>>     Well, updated a2dp-codec.h header will be needed, as there is no
>>>>> a2dp_aac_t in PA.
>>>>
>>>>
>>>> Correct.
>>>>
>>>>> 2) Handle codec capabilities negotiation
>>>>>      endpoint_set_configuration() seems to be responsible to send out
>>>>> the
>>>>> final
>>>>>      SetConfiguration AVDTP packet.
>>>>>      But which codec will bluez5 choose? Just highest codec number of
>>>>> both SINK and SOURCE
>>>>>      device?
>>>>>
>>>>>      endpoint_select_configuration() seems to be related to handle the
>>>>> response from the SINK
>>>>>      device.
>>>>
>>>>
>>>> Right again.
>>>>
>>>> Maybe get this working, and then we can worry about how to select
>>>> codecs. It'll probably have to be based on some module configuration
>>>> (which in turn will be guided by the h/w that this is running on).
>>>> Default could be something we select based on order of quality.
>>>>
>>>>>      But for multi-codec support, how do we distinguish one codec
>>>>> capability from another?
>>>>>
>>>>> 3) Record final codec profile into some structure of userdata in
>>>>> module-bluez5-devices.c
>>>>>
>>>>> 4) Call codec encode function in thread_func()
>>>>>
>>>>> Any comment is welcomed.
>>>>
>>>>
>>>> With GStreamer, either you'd probably do something like appsrc !
>>>> <encoder> ! <rtp payloader> ! appsink to run the encode, in
>>>> a2dp_process_render().
>>>
>>>
>>> The "either" was for a second option involving directly writing to an
>>> fdsink, but I think we should keep that in PA so coexisting with the
>>> current SBC code is easier.
>>
>>
>> While this is all possible since BlueZ, nor most of the stacks,
>> supports the so called multiplex mode using computing intensive codecs
>> is not desirable, in fact the only reason to support AAC and MP3
>> probably is in a passthrough mode where the source just send directly
>> the data without any transcoding, otherwise we would need to encode
>> all audio using those codecs, including system alerts, etc, which may
>> not be very nice for battery powered devices.
>
>
> Yes, extra encoding will reduce battery time of course.
> However IIRC iOS is using AAC codec if sink device support it, I think it's
> not without any reason.

Of course they use AAC it is the format they use in itunes.

>>
>> If someone would like a good alternative to SBC I would suggest using
>> OPUS, since that is used for VoIP and webrtc the codec should be
>> probably less computing intensive and it most likely more suitable for
>> live streams. gstreamer supports OPUS so it would probably make sense
>> to use it. Obviously, this would only work if both ends support OPUS,
>> and the stream initiator actually selects it to be used over SBC.
>
>
> Just as you mentioned, the problem is the sink side support.
> Despite some closed-source codec like AptX or LDAC, from my experience, AAC
> codec is more widely adopt than MP3.
>
> Several of my BT headphones from Bose QC35 to Sony MDR1000 support AAC but
> lack support for MP3.
>
> And I didn't even see any BT headphone with OPUS support.

Sure, though PA works as a sink as well, the only mandatory codec is SBC though.

> Thanks,
> Qu
>
>>
>> There is one last problem, despite some Android phones claiming to
>> have codec preference in practice there is no way to influence the
>> codec selection of the remote end since both ends can start a new
>> stream and over A2DP one can just enumerate codec capabilities. In
>> BlueZ we just assume the order of endpoints registration shall be used
>> as priority/preference, though we do this per process/D-Bus
>> connection.

-- 
Luiz Augusto von Dentz