gstreamer / Farstream / reSIProcate interoperability

[Olivier Crête]

Hi,

I think your model is the right one.

The local devices and file playback are intentionally  out of scope for Farstream.

The fsrtpconference element is largely a wrapper around rtpbin just like webrtcbin, the main functional difference is that Farstream also handles the encoding and decoding, while webrtcbin doesn't. You can definitely mix both in the same pipeline if we want. You may also want to have separate pipelines and use inter{audio,video}{sink,src} to transfer the frames between the pipelines if you want to decouple them a little.

Olivier

On April 13, 2021 11:26:57 a.m. EDT, Daniel Pocock <daniel at pocock.pro> wrote:
>
>Farstream and Gstreamer both look like interesting partners for
>reSIProcate
>
>In reSIProcate, we can do
>
>a) high level programming with librecon (Conversation Manager API) or,
>
>b) more fine-grained control over SIP using the libdum API
>
>My initial thought is to focus on (a) and create a very basic binary,
>similar to rtpproxy and mediaproxy but based on gstreamer+farstream.
>
>The high-level classes in librecon almost complement the classes in
>Farstream:
>
>
>FsConference — Interface for farstream conference elements
>- librecon: Conversation
>
>FsParticipant — A participant in a conference
>FsSession — A session in a conference
>- librecon: RemoteParticipant, but only supports one session, so it is
>like a combination of FsParticipant and FsSession
>- librecon: has other types of Participant, e.g. LocalParticipant for
>webcam/screen/mic/speakers
>
>FsStream — A stream in a session in a conference
>- librecon: MediaStream
>- it looks like FsStream represents both the RTP and RTCP, same as
>librecon MediaStream
>
>
>To make this work in reSIProcate, it looks like we need to split the
>Participant class into Participant and Session so that we have the same
>class hierarchy as Farstream, support for video, etc.
>
>librecon also has
>- MediaResourceParticipant (for playing a file into the Conversation)
>and
>- LocalParticipant (to represent hardware like the webcam, screen, mic
>and speakers)
>
>Sometimes the RemoteParticipant is regular SIP and sometimes it is
>WebRTC (SIP over WebSocket).  The reSIProcate stack supports both of
>these very well at the SIP level.  As Gstreamer webrtcbin is not part
>of
>Farstream, how does it relate to FsSession and FsStream?  Or can I just
>manage webrtcbin elements on my own and link the pads together from my
>own code?
>
>_______________________________________________
>gstreamer-devel mailing list
>gstreamer-devel at lists.freedesktop.org
>https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

-- 
Olivier Crête
olivier.crete at collabora.com