[pulseaudio-discuss] GSoC: Call for project ideas

[Tanu Kaskinen]

On Mon, 2013-03-25 at 13:23 +0000, Arun Raghavan wrote:
> On Sat, 2013-03-23 at 11:11 +0000, Toby Smithe wrote:
> > Hi,
> > 
> > I have a fairly free summer coming up, and thought it would be nice to
> > participate in GSoC. For a while I've been interested in PulseAudio, and
> > I have an idea for a project. I wonder if you might say whether you
> > think it plausible.
> > 
> > I use PulseAudio's native protocol streaming quite a lot, and I've
> > noticed that it seems quite rudimentary. I read the code a couple of
> > releases back, and it seems just to stream uncompressed PCM over
> > TCP. With a wireless connection and multi-channel audio, this quickly
> > becomes impractical, with drops and latency problems. A while ago, I
> > looked into implementing Opus compression for the network streams, but
> > never had a chance. I think Opus would make the ideal codec because it
> > is very flexible, recently ratified as an Internet standard, and can be
> > remarkably lightweight (according to the official benchmarks).
> > 
> > In doing these network audio, I might also be able to move on to
> > auxiliary tasks like improving the GUI tools for this use-case.
> > 
> > Do you think this might work?
> 
> Having means of doing non-PCM streaming would definitely be desirable.
> That said, though, I'm a bit wary of introducing codec and RTP
> dependencies in  PulseAudio (currently we have this at only one point,
> which is the Bluetooth modules - something I don't see a way around).
> 
> Now there are two main concerns:
> 
> 1. Codecs: choosing a codec is not simple. There are always potential
> reasons to look at one vs. the other - CPU utilisation, bandwidth, codec
> latency, quality, specific implementation (libav vs. reference
> implementation vs. hardware acceleration) and so on.
> 
> 2. RTP: as our usage gets more complicated, we're going to end up
> implementing and maintaining a non-trivial RTP stack, which is actually
> quite hard.
> 
> Deciding where to draw the line with regards to what does and does not
> belong in PulseAudio is a bit tricky, but in my mind, encoding/decoding
> should very much not be in PulseAudio because that beast inevitably gets
> more complicated as you try to do more, and there are others solving the
> problem system-wide.
> 
> RTP, I can see a case for it being in PulseAudio, but it is also
> complicated, and as with codecs, there are other places in the system
> where it gets more attention and maintenance.
> 
> The simplest idea I can think of to deal with this meaningfully is to
> wrap a sink/source around a GStreamer pipeline to offload all that work
> that we don't want to duplicate in PulseAudio .
> 
> On the sink side, we'd hook up to an appsrc to feed PCM data to a
> pipeline. The pipeline would take care of encoding, RTP packetisation
> and possibly a corresponding RTCP stream. This would allow codec
> selection to be flexible, and in the distant future, could even support
> taking encoded data directly.
> 
> On the source side, we'd hook up to an appsink, receiving PCM data from
> the pipeline. The pipeline would take care decoding whatever the format
> is, take care of RTCP and maybe more advanced features such as a jitter
> buffer and packet-loss concealment (all of this can be plugged in or
> not, depending on configuration).
> 
> Doing it this way means you're using a better RTP stack that gets
> attention from a number of other use cases (plus assorted related
> goodies) and support for multiple codecs.
> 
> Thoughts?

I like the idea of offloading stuff to GStreamer very much.

Do you follow GStreamer development closely? If I've understood
correctly, GStreamer hasn't supported rewinding in the past, has there
been any talk about that lately? Lack of rewinding support in any part
of the audio pipeline causes bigger or smaller problems. I don't think
it's a critical problem in this particular case, although rewind support
would be useful in this case too. If we would like to use GStreamer for
implementing filters, rewind support would be pretty mandatory.

-- 
Tanu