parallel message processing (Python)

[Thom Nichols]

Hello DBus community,

We've been enjoying DBus as an IPC mechanism in Python for some time now,
however we've recently experienced a wrinkle which I think is related to how
incoming DBus messages are dispatched.

In short, all DBus messages appear to be handled serially within the same
process, even though threads are using private connections.  If a dbus
method call takes a long time to return (i.e. a python function annotated
with dbus.service.method or signal,) it prevents any concurrent messages
(and I suspect outgoing dbus calls) from being processed.

Is there any way this can be changed to something more like a thread pool so
that multiple dbus messages can be handled in parallel?  Particularly
messages to different services should not interfere with each other.  I'm
not averse to coding it myself, but I'm looking for input as to whether or
not it might be possible from an architectural standpoint.

Certainly we can get around the problem by making all of our remote methods
and signal handlers immediately dispatch work to a thread or thread pool (so
that the dbus handler returns quickly,) but that won't work for synchronous
dbus method calls.  Certainly that also adds some code overhead that I'd
like to solve at a lower level if possible.

Attached is a script that demonstrates the problem.  (Also here:
http://pastebin.com/4mTh4Q9Y )  Certainly this script is running as a single
process which seems to defeat the point of IPC, but the concept (and
underlying issue) is still present regardless I believe.

Thanks in advance for any input.

-Thom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/dbus/attachments/20110121/ecfbc4bc/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dbus-method-test.py
Type: text/x-python
Size: 4847 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/dbus/attachments/20110121/ecfbc4bc/attachment.py>