Need guidance for implementing ice candidate pool

Mon Jun 7 22:47:03 UTC 2021

Hi,

You definitely need to create the nicesrc too (or you need to attach
the receive callback of libnice to some thread). 

Oh about freezing you're right, it's indeed about the pairs, I guess if
there is not remote peer, then it won't create pairs and it won't
apply.

You still need to make sure that those unused streams are not taken
into account for the global agent state though.

Olivier

On Mon, 2021-05-24 at 14:03 -0400, Sherrill Lam via gstreamer-devel
wrote:
> Hi Olivier,
> 
> I have some follow up questions regarding your advice.
> 
> I've tried creating GstWebRTCICEStream objects ahead of time and
> using them when gst_webrtc_ice_add_stream() was called. But when I
> tried to use these pre-created GstWebRTCICEStream objects to gather
> ICE candidates, I couldn't gather any reflexive candidates without
> gst_webrtc_ice_add_stream() being called. According to Libnice logs,
> component_io_cb() was not triggered until then.
> 
> I suspected it was because nicesrc element was not created thus
> component_io_cb() was not triggered. So I tried pre-creating
> TransportStream objects instead of GstWebRTCICEStream objects. And
> this time, reflexive candidates could be pre-gathered before
> gst_webrtc_ice_add_stream() being called. But I'm not sure if this is
> the correct direction to go, as we would need to properly configure
> the pre-created TransportStream objects so they could be used later
> (such as in _get_or_create_data_channel_transports() and
> _get_or_create_rtp_transport_channel()). Or maybe the
> original GstWebRTCICEStream approach would work and I just missed
> something. Any suggestions?
> 
> Also, I'm a bit confused about your suggestions with the
> freezing/unfreezing candidates. When I read the RFC for
> frozen candidates, my understanding is frozen candidates is a
> mechanism for candidate pairs, not for individual candidates. In our
> scenarios, ice candidate pooling is only used before SDP negotiation
> happens, which means there won't be any remote candidates at the
> point and no candidate pairs are created yet. Could you elaborate
> more on how the freezing/unfreezing candidates fit in the picture
> here?
> 
> Thanks a lot!
> Sherrill
> 
> On Sun, 9 May 2021 at 16:01, Sherrill Lam <sherrill.lam at gmail.com>
> wrote:
> > Hi Olivier and Mathieu,
> > 
> > Thanks a lot for your reply! When I think more more on it, I think
> > Olivier is right that the implementation is more a webrtc level
> > issue. I'm going to start from pre-allocating streams and caching
> > ice candidates, and see how it goes.
> > 
> > Thanks again for your great suggestions!
> > 
> > Regards,
> > Sherrill
> > 
> > On Mon, 3 May 2021 at 15:54, Olivier Crête via gstreamer-devel
> > <gstreamer-devel at lists.freedesktop.org> wrote:
> > > Hi,
> > > 
> > > Giving this a bit more though, I guess one could create those
> > > unallocated streams hidden inside libnice and expose them when
> > > nice_agent_add_stream() is called, but it's a bit trickier as
> > > that means forcing eveyr stream to have the same number of
> > > components, which isn't standard in ICE (but it's always true for
> > > webrtc as rtcp-mux is mandated now).
> > > 
> > > Olivier
> > > 
> > > On Mon, 2021-05-03 at 15:42 -0400, Olivier Crête via gstreamer-
> > > devel wrote:
> > > > Hi,
> > > > 
> > > > With my libnice maintainer hat on, it's definitely something I
> > > > would accept.
> > > > 
> > > > The tricky part is the implementation. At first, I though we'd
> > > > want to create a new object separate from the ICE agent to do
> > > > the gathering, but it gets quite tricky quickly as the
> > > > gathering and especially TURN code is quite tied to the ICE
> > > > agent code.
> > > > 
> > > > And looking at the spec, I think the implementation is mostly a
> > > > webrtc level issue. All that you need to do is to add a new API
> > > > to GstWebRTCICE to create GstWebRTCICEStream objects ahead of
> > > > time and "cache" them, then when gst_webrtc_ice_add_stream() is
> > > > called, you can just return one of the pre-allocated streams.
> > > > The only issue is that we need to add an API to libnice to mark
> > > > a stream "unallocated" so that it gets ignored when doing the
> > > > freezing/unfreezing dance, and the same thing needs to happen
> > > > inside webrtcbin so they get ignored when trying to compute the
> > > > overall agent state.
> > > > 
> > > > Olivier
> > > > 
> > > > 
> > > > On Mon, 2021-05-03 at 21:21 +0200, Mathieu Duponchelle via
> > > > gstreamer-devel wrote:
> > > > > Hey, this seems interesting, perhaps open an issue in
> > > > > libnice's gitlab to discuss this?
> > > > > 
> > > > > On 5/2/21 5:29 AM, Sherrill Lam via gstreamer-devel wrote:
> > > > > 
> > > > > > Hi there!
> > > > > > 
> > > > > > My team is interested to implement a prefetch ice candidate
> > > > > > pool and add an ice-candidate-pool-size property ( [rtc
> > > > > >
> > >
> >
> spec](https://www.w3.org/TR/webrtc/#dom-rtcconfiguration-icecandidatepoolsize
> > > > > > ) ) for webrtcbin element. We think it will help accelerate
> > > > > > the SDP generation at startup process. We’d like to get
> > > > > > more guidance on what would be the best way to do it.
> > > > > > 
> > > > > > Although ice candidate pool is a webrtc concept, I think it
> > > > > > might make more sense if the pool itself is implemented
> > > > > > within NiceAgent in Libnice. We could add an API in Libnice
> > > > > > to create such a pool with a specified pool size and start
> > > > > > generating ice candidates. When
> > > > > > nice_agent_gather_candidates() is called, it will first
> > > > > > check if there are candidates in the pool before generating
> > > > > > new ones.
> > > > > > 
> > > > > > Any thoughts or suggestions?
> > > > > > 
> > > > > > Thanks!
> > > > > > Sherrill
> > > > > > 
> > > > > > 
> > > > > > _______________________________________________
> > > > > > gstreamer-devel mailing list
> > > > > > gstreamer-devel at lists.freedesktop.org
> > > > > >
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > > > _______________________________________________
> > > > > gstreamer-devel mailing list
> > > > > gstreamer-devel at lists.freedesktop.org
> > > > >
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > > 
> > > > _______________________________________________
> > > > gstreamer-devel mailing list
> > > > gstreamer-devel at lists.freedesktop.org
> > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > 
> > > 
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > gstreamer-devel at lists.freedesktop.org
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

-- 
Olivier Crête
olivier.crete at collabora.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210607/98a5f34e/attachment-0001.htm>