[pulseaudio-discuss] [RFC - MP3 passthrough over A2DP 0/2]

[bossart.nospam at gmail.com]

From: Pierre-Louis Bossart <pierre-louis.bossart at intel.com>

Hi,
as an extension of the AC3/IEC958 passthrough over SPDIF/HDMI
functionality I contributed this summer, I tried to see if PulseAudio
could handle compressed data and pass it to a sink if ithe latter
supports the format natively. You will find below a patch showing a
proof of concept where you can send an MP3 file straight to a
Bluetooth headset, without having to decode the MP3 content and
re-encode in SBC. The functionality is similar to what existed before
with gstreamer's a2dpsink, except that now the routing is simplified
and it's easier to enable this power saving feature: all the audio
flows through PulseAudio, meaning all the routing is controlled in a
single location.

Full-disclosure: my long-term goal isn't really to work on Bluetooth,
but to enable hardware offload for low-power and audio/video use
cases. I just used this case as a non-proprietary example to see what
type of infrastructure we need to handle compressed data, once this is
available upstream then we can have additional silicon-specific
optimizations for other cases (eg when a DSP is available in the SOC).

Here's how to test the patch:
0. make sure you have mpeg sources enabled in your
/etc/blueooth/audio.conf or /usr/etc/bluetooth/audio.conf file.
1. make sure your headset supports mp3 with gst-launch filesrc
location=<file.mp3> ! mp3parse ! a2dpsink device=00:11:22:33:44
2. launch pulseaudio
3. launch pavucontrol
3. select the 'mp3 passthrough' configuration
4. send data swith commands such as 'paplay -v --raw --passthrough
--format=mp3 --rate=44100 --device=bluez_sink.00_0D_3C_B2_07_6B
--channels=2 file.mp3

There are a couple of known issues that will need to be fixed, but at
this time I would really like to get some feedback.

Trivial points:
- The parsing doesn't handle the case where the frame size is greater
than the mtu. 128kb/s should be fine, 320bp/s is known to break.
- the behavior when the synchronization is lost isn't quite working. I
jus thacked something to find out the length and duration of mp3
frames, but this would need more work to become bullet-proof.

Minor points
- I added an PA_MP3 format, but it might be a better option to create
an 'extended' format, and use a second data structure instead of the
'passthrough' flag. A single flag isn't very useful, a second data
structure 'pa_sample_ex' could contain as many parameters as needed
for other formats.
- The monitoring capability needs to be disabled for passthrough data,
I didn't find an elegant way to do this
- the concept of 'passthrough sink' needs to change. In the case of
AC3 data, the same sink could take either PCM or IEC958 data. Here we
need to choose between mp3 or pcm, you really need to separate sinks.
- The playback stops before all the data is rendered. This is an issue
for small files
- There's some on-going work to change the communication with bluez by
using the Media API, this code would be to be rewritten once the
changes are upstreamed.

important issues:
- As in the AC3 case, there's a need to reconfigure the sink sampling
frequency when the sink-input and sink are linked. PulseAudio
traditionally operatates at a single default frequency, this
limitation does not apply for compressed data since we cannot mix.
- Somehow zeroed samples are sent to the sink even when no sink-input
is linked, as soon as the sink becomes active. This doesn't make any
sense for compressed data, we'd need to suspend and restart when new
data is available
- The sink capabilities need to be exposed to the application, so that
it can choose to decode and send PCM data or send compressed data to
the hardware.
- Instead of kill the client when the sink-input is moved to a
different sink, one should inform the application of the capabilities
changes and give it time to instantiate a decoder.
- And I saved the most difficult for the end, the whole concept of
time management and latency control is by construction broken when
handling compressed data. You don't have a direct relationship between
number of bytes and usec, something different is needed. In all cases
when passing compressed content to a hardware receiver, there's an
explicit time representation (sample counts or usec), so instead of
finding the time with GET_LATENCY messages, one could send a GET_TIME
message to retrieve the information. Again since there's no mixing,
there's no difference between the stream time and the sink time.

Handling compressed data has been on the PulseAudio roadmap for quite
some time, I hope this patch generates a lot of comments.
Acknowledgements to Stefan Kost and Luiz von Dentz who provided very
useful support while I was fighting with my headset, I couldn't have
completed this first pass without their help.
Thanks for your feedback.

Pierre Bossart
Intel Corporation, Ultra Mobility Group

Pierre-Louis Bossart (2):
  mp3 passthrough: core changes
  mp3 passthrough: Bluetooth changes

 src/modules/bluetooth/module-bluetooth-device.c |  861 +++++++++++++++++++----
 src/modules/bluetooth/rtp.h                     |   14 +-
 src/pulse/sample.c                              |   18 +-
 src/pulse/sample.h                              |    3 +
 src/pulsecore/envelope.c                        |    2 +-
 src/pulsecore/mime-type.c                       |    1 +
 src/pulsecore/sample-util.c                     |    6 +
 src/pulsecore/sink-input.c                      |   26 +-
 src/pulsecore/source-output.c                   |    4 +
 9 files changed, 780 insertions(+), 155 deletions(-)

--
1.7.2.3