[pulseaudio-discuss] Working on a new network transport for PulseAudio

[Tanu Kaskinen]

On Mon, 2019-05-06 at 21:35 +0400, Victor Gaydov wrote:
> Hi,
> 
> I'm working on Roc, a new project implementing real-time streaming over
> network[1].
> 
> Currently it can stream PCM using RTP + FECFRAME (Reed-Solomon and
> LDPC-Staircase are supported, using OpenFEC library). There are plans to
> add support for more encodings (including Opus) and protocols (including
> SAP/SDP).
> 
> Among other things, we've implemented a couple of PulseAudio modules
> that use Roc as a network transport. I've written an overview post
> about the project and these modules[2].
> 
> Both Roc and PA modules are in early stage of development. PA modules
> still miss some essential features, e.g. there is no latency reporting
> and service discovery.
> 
> I'd like to discuss the opportunity of submitting those modules to the
> PA upstream in future. Some thoughts?
> 
> [1] https://github.com/roc-project/roc
> [2] https://gavv.github.io/articles/new-network-transport/
> 
> -- Victor

Miscellaneous thoughts in random order:

At least I'm in favor of accepting the new modules in PulseAudio, if
they provide less glitches than the tunnel modules or the existing RTP
modules. And I'd love to have the adaptive resampling logic maintained
in an external library by someone who knows how it works (I don't
understand and I don't look forward to trying to learn how the
resampling code in the existing RTP modules works).

I should probably mention that if you submit patches, I can't promise a
timely review - there's too much work that I'm already committed to.

Arun has in the past talked about some plans for reworking the RTP
stuff to use GStreamer - I hope that won't be an impediment for
accepting these modules.

Are there any plans to support multicast? I don't know how important
that is to people, it's just that I think the existing RTP modules use
multicast by default (which tends to break wireless networks...)

Can one receiver connect to multiple senders? If there are two client
machines that want to use speakers connected to a single receiver
machine, does module-roc-sink-input have to be loaded twice (and with
different ports)?

Any plans for dynamic latency based on stream requests? If the goal is
to support low latency, it's probably desirable to have a bit larger
latency for music playback, and reduce the latency on the fly when a
low-latency application starts.

How much of the parameters could theoretically be changed on the fly (I
say theoretically, because you probably haven't implemented any such
controls yet)? Imagine a GUI for configuring this system - trying
different resampler profiles or latency settings on the fly could be
useful. Changing the IP address or port probably doesn't need to be
doable mid-stream.

People have requested in the past that the tunnel sink would change the
stream sample rate and format on the fly based on the application's
stream, similar to how the hardware sink supports automatic sample rate
switching. Would this be doable? It doesn't need to (and indeed
shouldn't) be done mid-stream, but if nothing is currently using the
Roc sink and a new stream appears, it would be good to configure the
network stream parameters based on the application stream.

The blog post has this sentence: "Roc performs the clock rate
adjustment on the receiver, while PulseAudio does it on the sender" -
that's not true. Doing the adjustment on the sender would require some
kind of feedback system from the receiver to the sender, and that
doesn't exist. PulseAudio does the adjustment on the receiver.

What happens if the connection dies temporarily (receiver reboots or
crashes, or there's a network outage)? Can the system automatically
recover when the receiver comes back online?

Can the system deal with the hardware sink changing on the fly? There
will be a discontinuity in the reported latency. Similarly, can the
system deal with the user changing the latency offset of a port? That
too will cause a strange jump in latency reports.

Can the system deal with the hardware sink suspending? The Roc sink
input will be connected to the hardware sink, but the sink won't be
asking for more data until the sink resumes.

Based on the great documentation, Roc looks like a quality product!

-- 
Tanu

https://www.patreon.com/tanuk
https://liberapay.com/tanuk