[pulseaudio-discuss] GSoC: Call for project ideas

[Arun Raghavan]

On Sat, 2013-03-23 at 11:11 +0000, Toby Smithe wrote:
> Hi,
> 
> I have a fairly free summer coming up, and thought it would be nice to
> participate in GSoC. For a while I've been interested in PulseAudio, and
> I have an idea for a project. I wonder if you might say whether you
> think it plausible.
> 
> I use PulseAudio's native protocol streaming quite a lot, and I've
> noticed that it seems quite rudimentary. I read the code a couple of
> releases back, and it seems just to stream uncompressed PCM over
> TCP. With a wireless connection and multi-channel audio, this quickly
> becomes impractical, with drops and latency problems. A while ago, I
> looked into implementing Opus compression for the network streams, but
> never had a chance. I think Opus would make the ideal codec because it
> is very flexible, recently ratified as an Internet standard, and can be
> remarkably lightweight (according to the official benchmarks).
> 
> In doing these network audio, I might also be able to move on to
> auxiliary tasks like improving the GUI tools for this use-case.
> 
> Do you think this might work?

Having means of doing non-PCM streaming would definitely be desirable.
That said, though, I'm a bit wary of introducing codec and RTP
dependencies in  PulseAudio (currently we have this at only one point,
which is the Bluetooth modules - something I don't see a way around).

Now there are two main concerns:

1. Codecs: choosing a codec is not simple. There are always potential
reasons to look at one vs. the other - CPU utilisation, bandwidth, codec
latency, quality, specific implementation (libav vs. reference
implementation vs. hardware acceleration) and so on.

2. RTP: as our usage gets more complicated, we're going to end up
implementing and maintaining a non-trivial RTP stack, which is actually
quite hard.

Deciding where to draw the line with regards to what does and does not
belong in PulseAudio is a bit tricky, but in my mind, encoding/decoding
should very much not be in PulseAudio because that beast inevitably gets
more complicated as you try to do more, and there are others solving the
problem system-wide.

RTP, I can see a case for it being in PulseAudio, but it is also
complicated, and as with codecs, there are other places in the system
where it gets more attention and maintenance.

The simplest idea I can think of to deal with this meaningfully is to
wrap a sink/source around a GStreamer pipeline to offload all that work
that we don't want to duplicate in PulseAudio .

On the sink side, we'd hook up to an appsrc to feed PCM data to a
pipeline. The pipeline would take care of encoding, RTP packetisation
and possibly a corresponding RTCP stream. This would allow codec
selection to be flexible, and in the distant future, could even support
taking encoded data directly.

On the source side, we'd hook up to an appsink, receiving PCM data from
the pipeline. The pipeline would take care decoding whatever the format
is, take care of RTCP and maybe more advanced features such as a jitter
buffer and packet-loss concealment (all of this can be plugged in or
not, depending on configuration).

Doing it this way means you're using a better RTP stack that gets
attention from a number of other use cases (plus assorted related
goodies) and support for multiple codecs.

Thoughts?

-- Arun