[pulseaudio-discuss] Working on a new network transport for PulseAudio

[Victor Gaydov]

On Fri, 10 May 2019 21:15:48 +0300
Tanu Kaskinen <tanuk at iki.fi> wrote:

> On Mon, 2019-05-06 at 21:35 +0400, Victor Gaydov wrote:
> > Hi,
> > 
> > I'm working on Roc, a new project implementing real-time streaming
> > over network[1].
> > 
> > Currently it can stream PCM using RTP + FECFRAME (Reed-Solomon and
> > LDPC-Staircase are supported, using OpenFEC library). There are
> > plans to add support for more encodings (including Opus) and
> > protocols (including SAP/SDP).
> > 
> > Among other things, we've implemented a couple of PulseAudio modules
> > that use Roc as a network transport. I've written an overview post
> > about the project and these modules[2].
> > 
> > Both Roc and PA modules are in early stage of development. PA
> > modules still miss some essential features, e.g. there is no
> > latency reporting and service discovery.
> > 
> > I'd like to discuss the opportunity of submitting those modules to
> > the PA upstream in future. Some thoughts?
> > 
> > [1] https://github.com/roc-project/roc
> > [2] https://gavv.github.io/articles/new-network-transport/
> > 
> > -- Victor  
> 
> Miscellaneous thoughts in random order:
> 
> At least I'm in favor of accepting the new modules in PulseAudio, if
> they provide less glitches than the tunnel modules or the existing RTP
> modules. And I'd love to have the adaptive resampling logic maintained
> in an external library by someone who knows how it works (I don't
> understand and I don't look forward to trying to learn how the
> resampling code in the existing RTP modules works).

Great!

> I should probably mention that if you submit patches, I can't promise
> a timely review - there's too much work that I'm already committed to.

OK.

> Arun has in the past talked about some plans for reworking the RTP
> stuff to use GStreamer - I hope that won't be an impediment for
> accepting these modules.

Interesting. This should be discussed, I think, because Roc
features overlap with the network part of GStreamer.

> Are there any plans to support multicast? I don't know how important
> that is to people, it's just that I think the existing RTP modules use
> multicast by default (which tends to break wireless networks...)

Multicast is in our roadmap, but I'd like to work on it a bit later,
after implementing some more basic stuff like receiver feedback,
session negotiation, and compression.

> Can one receiver connect to multiple senders? If there are two client
> machines that want to use speakers connected to a single receiver
> machine, does module-roc-sink-input have to be loaded twice (and with
> different ports)?

Single receiver handles multiple senders connected to it (and mixes
their streams).

> Any plans for dynamic latency based on stream requests? If the goal is
> to support low latency, it's probably desirable to have a bit larger
> latency for music playback, and reduce the latency on the fly when a
> low-latency application starts.

There are plans to implement automatic latency adjustment (depending on
the network conditions) on the receiver side.

Allowing the user to adjust the latency manually on the sender side
sounds reasonable but we need to think what protocols are suited for
that. Probably RTSP, which is in our roadmap.

> How much of the parameters could theoretically be changed on the fly
> (I say theoretically, because you probably haven't implemented any
> such controls yet)? Imagine a GUI for configuring this system - trying
> different resampler profiles or latency settings on the fly could be
> useful. Changing the IP address or port probably doesn't need to be
> doable mid-stream.

Currently it's not possible to change anything on fly, except binding
new ports.

There are plans to implement automatic adjustment of latency, FEC block
size, packet size, and bitrate. This means that in future all these
parameters will be configurable on fly at least internally. And so it
would be possible to make them configurable via API too.

> People have requested in the past that the tunnel sink would change
> the stream sample rate and format on the fly based on the
> application's stream, similar to how the hardware sink supports
> automatic sample rate switching. Would this be doable? It doesn't
> need to (and indeed shouldn't) be done mid-stream, but if nothing is
> currently using the Roc sink and a new stream appears, it would be
> good to configure the network stream parameters based on the
> application stream.

I think it would be possible after we'll add some sort of session
negotiation (RTSP or SIP). I'm planning to start working on it soon.

> The blog post has this sentence: "Roc performs the clock rate
> adjustment on the receiver, while PulseAudio does it on the sender" -
> that's not true. Doing the adjustment on the sender would require some
> kind of feedback system from the receiver to the sender, and that
> doesn't exist. PulseAudio does the adjustment on the receiver.

My bad. Fixed.

> What happens if the connection dies temporarily (receiver reboots or
> crashes, or there's a network outage)? Can the system automatically
> recover when the receiver comes back online?

Currently things work very simple. Sender just sends stream to the
given address and has no feedback from the receiver. Receiver
automatically creates a session when it gets a packet from a new sender
and removes the session when there are no packets during a timeout. So
yes, the system will automatically recover.

Things will become a bit more complicated when we'll implement session
negotiation. We'll have to re-negotiate the session after an outage.

> Can the system deal with the hardware sink changing on the fly? There
> will be a discontinuity in the reported latency. Similarly, can the
> system deal with the user changing the latency offset of a port? That
> too will cause a strange jump in latency reports.

Currently no (I didn't test it yet).

> Can the system deal with the hardware sink suspending? The Roc sink
> input will be connected to the hardware sink, but the sink won't be
> asking for more data until the sink resumes.

Currently no (I have tested it; sink suspending breaks the receiver).

> Based on the great documentation, Roc looks like a quality product!

Thanks :)

-- Victor