KWIG Qt->Gtk porting layer and merging main loops.

[Magnus Bergman]

On Mon, 01 Nov 2004 20:52:25 -0500
Owen Taylor <otaylor at redhat.com> wrote:

> On Tue, 2004-11-02 at 00:59 +0100, Magnus Bergman wrote:
> > > Quick summary:
> > > 
> > >  libevent is seriously too simple:
> > > 
> > >   - Not thread safe
> > 
> > True. But I'm not convinced that it's impossible to implement thread
> > safety on top libevent quite easily. But better yet, libevent could
> > be fixed to handle threads.
> 
> Thread safety means being to do things like have an event loop running
> in one thread and add an idle function in another thread, and have the
> other thread notice the new idle and fire it at the earliest 
> opportunity.
>  
> > >   - Doesn't support multiple independent main contexts
> > 
> > Couldn't that be implemented on top of one main context? Perhaps it
> > would be inefficient (I'm not familiar with the exact implementation
> > of glib) but I'm not convinced it would be impossible.
> 
> It would be very hard to implement it on top of one main context and
> keep things efficient. Consider two threads, with two main contexts,
> each with one fd, where the main loops are run
> 
>   1      A ---------------------------- C
>   2                   B ------------------------------ D
> 
> At point A you have to add fd1 to the libevent, at point B add fd2, at
> point C remove fd1, at point D remove fd2.
> 
> That's a little inefficient. Imagine if each context had 100 fds?
> You can't just leave all the fds in the poll all the time, because 
> the poll will be woken up prematurely if there is data waiting in
> an active fd.

I see. I was aware of the problem of having separate sets of events
running in different threads, but my idea for a solution is aperantly
different (probably also inferior) to the one used in Glib. My idea was
that if each event be belongs to a context and and every thread that
makes a call to the main loop also belongs to a context. And then an
event happens it is checked thatever the context has any theads,
otherwise it is discarded. This works for me. But honestly I've never
thought about the need to mass-add and mass-remove sets of events in an
efficient way. If this is a requirement from Glib, then of course it's a
requirement for a theoretical underlying event library too.

> > >   - Fails the "can you write an X event source as a callback" test
> > >     (Requires special handling because XPending() can become true
> > >     between calls to select())
> > 
> > I don't know what that's about, but I assume it's an API that
> > requires expicit polling (reading the state of some shared memory
> > perhaps?). But as long as it isn't something beyond my imagination
> > this could be implemented using a timer with libevent as well as
> > using timeout to select()/poll() directly, without libevent. (Or
> > perhaps using the alarm signal.) For there isn't really an
> > alternative to way do it, is there?
> 
> Somewhat simplified, XLib has three relevant primitives:
> 
>  XPending() - are there events waiting
>  XNextEvent() - give me the next event
>  ConnectionNumber() - give me the file descriptor for incoming events
> 
> The thing you have to deal with is that XLib has an internal queue
> of events. So, polling on ConnectionNumber() can block even if 
> XPending() would return TRUE. The event loop needs to make sure it
> calls XPending() before starting to poll. 

So XPending() needs to be called reguarly, right? Isn't a timer event
that calls it reguarly and adds the fd from ConnectionNumber() to the
main loop then it returns TRUE an appropriate solution? I still don't
get it.

> Could you redesign XLib to prevent this problem? Yes, add a callback
> when XPending goes from FALSE to TRUE. But that's not really an
> option we have in general.
> 
> There are workarounds you could come up with, but they all are going
> to cause considerably performance damage to the central operation of
> a GUI app. And XLib isn't the only library with this type of API.
> 
> > >  I'm not arguing that you need the level API complexity of
> > >  GMainLoop, but anything that wouldn't allow that level of
> > >  complexity to be be built on top isn't usable.
> > 
> > True. But the only thing that needs to be common is the call to
> > select()/poll() and a way to register low level events (the three
> > types of them). Completely different things could be implemented on
> > top of this and work together nicely, which is the one and only
> > problem that needs to be solved, right?
> 
> I don't think so. Let me describe just one of the other issues you
> would face - priorities. GLib can have priorities where, for two, say,
> file descriptor watch, the lower priority one will never be called as
> long as there is data on the higher priority.
> 
> Building GLib on top of a low-level libevent isn't hard basically, you
> end up treating the underlying main loop as a user space poll(), and
> set flags from callbacks. Using other libevent programs with this GLib
> main loop works "OK" ... you just get side effects when GLib calls the
> pseudo-poll.
> 
> But the other way doesn't work well. Another app runs the main loop,
> and GLib gets a callback from libevent for fd_low. Without knowing the
> poll() results for fd_high, GLib doesn't know whether to call the
> application.

Yes, I get that now. I had a solution for it but realize that Glib has a
much better one.

> Does that mean that you can't build GLib on top of a simpler loop? No.
> But it's going to have to export something more like the GLib GSource
> API with the prepare/check/dispatch stages than a simple "add a
> callback for this API"
> 
> GLib's main loop has a lot of careful thinking in it making it a
> universal main loop. I suspect a good libevent that really met all the
> necesary requirements would look a lot like GMainLoop, stripped of the
> GLib and without some of the design flaws (e.g. g_main_looop_query())
> 
> I guess the onus is on me to write up something explaining GMainLoop
> from a main loop designers point of view. :-)

I've come to realize that Glib has features missing from libevent that I
know about the need of. But on the other hand libevent has support for
/dev/poll, epoll and kqueue in addition to select() and poll(), which is
somthing I believe Glib lacks. It would be great if one could have the
benefits from both...