QUIC Transport architecture for GStreamer
Samuel Hurst
samuelh at rd.bbc.co.uk
Thu Feb 16 14:55:49 UTC 2023
Hello all,
TL;DR: How best to implement a bidirectional transport src/sink in
GStreamer?
I've been looking at media transport over QUIC transport [1] for quite
some time, and I would like to start implementing some things in
GStreamer, however I'm unsure as to exactly what architecture I should
go with and would appreciate some advice from some GStreamer experts.
My initial thoughts involved writing a quicsrc/quicsink element that
would itself sit between the rest of the pipeline and a pair of udpsrc
and dynudpsink elements. QUIC Datagrams can be passed on sometimes pads,
with more request pads created dynamically for each QUIC stream and
announced via named signals.
In the example below, an rtpquicdemux element would manage the QUIC
datagram and stream flows to enable reception of RTP over QUIC [2],
passing regular RTP packets down to existing GStreamer RTP elements:
+--------+ +---------+ +---------+
| udpsrc +--> | | | +-------------+ +-----------+
+--------+ | | | +-> rtp...depay +-> decodebin +->
| quicsrc +----> rtpquic | +-------------+ +-----------+
+--------+ | +----> demux | +-------------+ +-----------+
| dynudp <--+ | | +-> rtp...depay +-> decodebin +->
| sink | | | | | +-------------+ +-----------+
+--------+ +---------+ +---------+
However, in the case where you need to send and receive on the same QUIC
transport connection, this model seems to conceptually break down, as
the quicsrc and quicsink elements wouldn't be able to share the QUIC
transport connection, which doesn't make sense for a bidirectional media
session such as a video call:
+--------+ +---------+ +----------+ +--------+
| udpsrc +--> | | +-> dynudp |
+--------+ | +-> | | | sink |
| quicsrc | ... --> quicsink | +--------+
+--------+ | +-> | | +--------+
| dynudp <--+ | --> <-+ udpsrc |
| sink | | | | | +--------+
+--------+ +---------+ +----------+
In addition, any bidirectional streams would need sink pads on the
quicsrc element to pass messages back, which sounds quite messy to me.
Instead, I wondered whether or not trying to incorporate the udpsrc and
dynudpsink elements was just over-complicating it for myself, and
instead I should just have the quicsrc and quicsink elements be
standalone, with a shared QUIC library (not element, but difficult to
show the difference in asciiart) between them managing the socket and
the QUIC transport connection transparently?
+---------+
| sockets |
+----^----+
|
+----v----+
+------------------> quiclib <-------------------+
| +---------+ |
+----v----+ +-------+ +-----+ +------+ +-----+ |
| +-->rtpquic+-> rtp +->decode+->audio| +----v-----+
| | | demux | |depay| | bin | |sink | | |
| quicsrc | +-----+-+ +----++ +--+-+-+ +---+-+ | |
| | +-----+ +-+----+ ++--+ +-+-+---+ | quicsink |
| | |audio+->encode+->rtp+->rtpquic+--> |
+---------+ | src | | bin | |pay| | mux | | |
+-----+ +------+ +---+ +-------+ +----------+
The above architecture is what happens in things like the httpsrc
elements which own the socket internally. I'd imagine the "quiclib"
would effectively be a singleton instance that would mediate all QUIC
transport connections in a given pipeline, in case there are multiple
sources/sinks, such as in an RTP mixer. However, this has challenges of
its own in terms of which element "owns" each connection, but that's a
problem for later on.
What would you suggest is most inkeeping with the intended GStreamer
design ethos, and isn't going to risk me tripping myself up down the line?
Best regards,
Sam
--
[1]: https://www.rfc-editor.org/rfc/rfc9000.html
[2]: https://datatracker.ietf.org/doc/draft-ietf-avtcore-rtp-over-quic/
More information about the gstreamer-devel
mailing list