QUIC Transport architecture for GStreamer

[Sanchayan Maity]

Hello,

Won't comment on the architecture bits, will leave that to the more
experienced folks than me.

Have posted a MR for a quicsrc/sink to gst-plugins-rs which might interest
you.
https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1036

It is pretty basic right now but will be happy to work on improvements to
perhaps enable some of the below use cases.

On 23-02-16 14:55:49, Samuel Hurst via gstreamer-devel wrote:
> Hello all,
> 
> TL;DR: How best to implement a bidirectional transport src/sink in
> GStreamer?
> 
> I've been looking at media transport over QUIC transport [1] for quite some
> time, and I would like to start implementing some things in GStreamer,
> however I'm unsure as to exactly what architecture I should go with and
> would appreciate some advice from some GStreamer experts.
> 
> My initial thoughts involved writing a quicsrc/quicsink element that would
> itself sit between the rest of the pipeline and a pair of udpsrc and
> dynudpsink elements. QUIC Datagrams can be passed on sometimes pads, with
> more request pads created dynamically for each QUIC stream and announced via
> named signals.
> 
> In the example below, an rtpquicdemux element would manage the QUIC datagram
> and stream flows to enable reception of RTP over QUIC [2], passing regular
> RTP packets down to existing GStreamer RTP elements:
> 
> +--------+  +---------+    +---------+
> | udpsrc +-->         |    |         | +-------------+ +-----------+
> +--------+  |         |    |         +-> rtp...depay +-> decodebin +->
>             | quicsrc +----> rtpquic | +-------------+ +-----------+
> +--------+  |         +---->  demux  | +-------------+ +-----------+
> | dynudp <--+         |    |         +-> rtp...depay +-> decodebin +->
> |  sink  |  |         |    |         | +-------------+ +-----------+
> +--------+  +---------+    +---------+
> 
> However, in the case where you need to send and receive on the same QUIC
> transport connection, this model seems to conceptually break down, as the
> quicsrc and quicsink elements wouldn't be able to share the QUIC transport
> connection, which doesn't make sense for a bidirectional media session such
> as a video call:
> 
> +--------+  +---------+        +----------+ +--------+
> | udpsrc +-->         |        |          +-> dynudp |
> +--------+  |         +->      |          | |  sink  |
>             | quicsrc |  ... --> quicsink | +--------+
> +--------+  |         +->      |          | +--------+
> | dynudp <--+         |      -->          <-+ udpsrc |
> |  sink  |  |         |        |          | +--------+
> +--------+  +---------+        +----------+

Was wondering if something like the socket property on udpsink/src or agent on
nicesrc/sink could work here.

> 
> In addition, any bidirectional streams would need sink pads on the quicsrc
> element to pass messages back, which sounds quite messy to me.

Hmm not sure about this with the above approach.

> 
> Instead, I wondered whether or not trying to incorporate the udpsrc and
> dynudpsink elements was just over-complicating it for myself, and instead I
> should just have the quicsrc and quicsink elements be standalone, with a
> shared QUIC library (not element, but difficult to show the difference in
> asciiart) between them managing the socket and the QUIC transport connection
> transparently?
> 
>                         +---------+
>                         | sockets |
>                         +----^----+
>                              |
>                         +----v----+
>      +------------------> quiclib <-------------------+
>      |                  +---------+                   |
> +----v----+  +-------+ +-----+ +------+ +-----+       |
> |         +-->rtpquic+-> rtp +->decode+->audio|  +----v-----+
> |         |  | demux | |depay| | bin  | |sink |  |          |
> | quicsrc |  +-----+-+ +----++ +--+-+-+ +---+-+  |          |
> |         |    +-----+ +-+----+ ++--+ +-+-+---+  | quicsink |
> |         |    |audio+->encode+->rtp+->rtpquic+-->          |
> +---------+    | src | | bin  | |pay| |  mux  |  |          |
>                +-----+ +------+ +---+ +-------+  +----------+
> 
> The above architecture is what happens in things like the httpsrc elements
> which own the socket internally. I'd imagine the "quiclib" would effectively
> be a singleton instance that would mediate all QUIC transport connections in
> a given pipeline, in case there are multiple sources/sinks, such as in an
> RTP mixer. However, this has challenges of its own in terms of which element
> "owns" each connection, but that's a problem for later on.
> 
> What would you suggest is most inkeeping with the intended GStreamer design
> ethos, and isn't going to risk me tripping myself up down the line?
> 
> Best regards,
> Sam
> 
> --
> [1]: https://www.rfc-editor.org/rfc/rfc9000.html
> [2]: https://datatracker.ietf.org/doc/draft-ietf-avtcore-rtp-over-quic/

-- 
Regards,
Sanchayan.