Fwd: disappearing messages with dbus_connection_read_write_dispatch & dbus_connection_pop_message

[Tim Wuyts]

Dear Havoc,

i've found the problem:
When you call dbus_connection_read_write(-1) in conjunction with
dbus_connection_pop_message(), you should pop all messages from the
queue before calling dbus_connection_read_write(-1) again. If you
don't, weird things happen.

I had my 'Eureka' moment when I saw this in the verbose output:

Message 0x804de68 (4 /org/freedesktop/DBus org.freedesktop.DBus
NameAcquired 's') removed from incoming queue 0x804c748, 2 incoming

indicating that the popped message was not for my app, but had a
'/org/freedesktop/DBus' path, and that there was more than 1 incoming
msg in the queue.

Thanks again for your insights.

Kind regards,
Tim

On 10/17/07, Havoc Pennington <hp at redhat.com> wrote:
> Hi,
>
> Tim Wuyts wrote:
> > From the documentation, I would think it should simply be a matter of
> > changing the timeout in the read_write to -1 and nothing else.
> > Unfortunately that simply doesn't work: messages are still being
> > 'lost'.
>
> That should work, afaik.
>
> If you built dbus with verbose mode enabled, then you can set
> DBUS_VERBOSE=1 when running your app and see if there are any clues.
>
> > It's rather worrying that this behavior is not consistent:
> > sometimes messages get through, sometimes they don't.
>
> That is not too surprising, because the behavior depends on timing -
> what bytes happen to be available to read when read_write() is called.
>
> > Which brings me to my second question: using read_write_dispatch(-1)
> > and then using borrow_message, it all works flawlessly. This seems
> > unlogical: if the 'dispatch' part would drop the message, then there
> > wouldn't be anything to 'borrow' afterwards, would there?
>
> I don't really know why this happens, but maybe you can dig into it.
>
> > And what really happens to the message in the queue after I return
> > (not steal) it? Does it get discarded (i.e. free'd), or does it keep
> > on consuming memory ad infinitum?
>
> When you return_message after borrowing it, it just gets dispatched
> normally. (If you ever call dispatch()). Which means any handlers are
> run then the message is freed.
>
> > Sorry for all these questions, but I'm trying to get a grip on the way
> > dbus works: we're using dbus on an embedded system, so there can't be
> > any missing messages or memory leaks.
>
> If you have not read the "Detailed description" at the top of
> http://dbus.freedesktop.org/doc/api/html/group__DBusConnection.html ,
> that may help. Or just look at the dbus-connection.c source code:
> http://dbus.freedesktop.org/doc/api/html/dbus-connection_8c-source.html
>
> There is nothing fundamentally complex, though the source has a bunch of
> goo related to thread locking. Messages arrive on a socket; as they are
> read from a socket, they go in a linked list (incoming queue). When you
> dispatch(), the next message in the queue is popped off, handlers
> invoked, then the message is freed. When you pop_message(), the next
> item in the queue is popped off. When you borrow_message(), the next
> item is popped off but you are expected to put it back and other threads
> are blocked from dispatching or popping anything until you do put it back.
>
> If you both dispatch sometimes and pop_message() sometimes, then it's
> essentially random whether the handlers get run by dispatch, or you get
> to pop the message. It just depends on timing of bytes arriving on the
> socket and when they happen to be read.
>
> Havoc
>