[libnice] Using (abusing?) libnice's event loop for outgoing traffic

[Olivier Crête]

Hi,

I would just get rid of the send thread and the GASyncQueue completely.
  Sending in libnice is non-blocking, so you should be able to just
call the nice_agent_send() at the place where you would normally put
the things in a queue.

Olivier

On Wed, 2018-05-09 at 15:50 +0200, Lorenzo Miniero wrote:
> Hi all,
> 
> as you may know, I'm using libnice in Janus, a WebRTC server. The way
> it's used right now is with two different threads per ICE agent: one
> runs the agent's GMainContext+GMainLoop, and as such is responsible
> for notifying the application about incoming events and packets via
> the libnice callback; another thread handles outgoing traffic, with a
> GAsyncQueue to queue packets, prepare them and shoot them out via
> nice_agent_send() calls.
> 
> These past few days I've been playing with an attempt to actually put
> those two activities together in a single thread, which would
> simplify
> things (e.g., in terms of locking and other things) and, ideally,
> optimize resources (we'd only spawn half the threads we do now). To
> do
> so, I decided to try and re-use the agent's event loop for that, and
> followed an excellent blog post by Philip Withnall to create my own
> GSource for the purpose:
> https://tecnocode.co.uk/2015/05/05/a-detailed-look-at-gsource/
> This was very helpful, as I ended up doing something very similar,
> since I was already using a GAsyncQueue for outgoing media myself.
> 
> Anyway, while this "works", the outgoing media is quite delayed when
> rendered by the receiving browser. I verified that media from clients
> does come in at the correct rate and with no delays there, by
> redirecting the traffic to a monitoring gstreamer pipeline, which
> means somehow the outgoing path is to blame. I "tracked" packets both
> using wireshark and log lines (e.g., when they were queued and when
> they were dispatched by the GSource), and did notice that some
> packets
> are handled fast, while others much later, and there are "holes" when
> no packet is sent (for ~500ms at times).  This shouldn't be the code
> failing to keep up with the things to do in time (since there's SRTP,
> there's encryption involved for every packet), as the CPU usage is
> always quite low.
> 
> At first I thought this could be ascribed to GSource priorities, but
> even playing with those nothing changed. One thing I noticed, though,
> was that the delay was quite more apparent in sendonly connections
> than in sendrecv ones. This means that, while in an EchoTest demo it
> is barely noticeable (even though still worse than with the two
> threads), in the VideoRoom demo, where you have monodirectional
> agents
> (some used to just publish media, others just to subscribe), it is
> much more evident (~600ms). Considering that the publisher's media is
> fine (as confirmed by the gstreamer monitor mentioned above), the
> only
> explanation I came up with was that, while in a bidirectional
> communication there's a lot of incoming traffic, on a subscriber's
> connection you only get occasional RTCP packets from time to time. In
> case there's some sort of timed poll deep in the libnice code, due to
> a single thread processing both directions this might mean having the
> event loop kept busy waiting for a file descriptor to generate an
> event, while outgoing packets pile up in the queue.
> 
> Does this make sense, and might this indeed be what's happening? Is
> what I'm trying to do indeed feasible or are there better ways to try
> and do it properly, e.g., via nice_agent_recv_messages instead of the
> recv callback? This is my first attempt at using Glib events in a
> more
> "conversational" way than for sporadic events, and so I realize I may
> be doing some rookie mistakes here.
> 
> Thanks!
> Lorenzo
> _______________________________________________
> nice mailing list
> nice at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/nice
-- 
Olivier Crête
olivier.crete at collabora.com