max_outgoing_bytes: What if a D-Bus peer is too slow to read?

[Alban Crequy]

Le Tue, 5 Oct 2010 13:31:11 -0400,
Havoc Pennington <hp at pobox.com> a écrit :

> Hi,
> 
> On Tue, Oct 5, 2010 at 11:56 AM, Alban Crequy
> <alban.crequy at collabora.co.uk> wrote:
> > In an implementation of the bus in the kernel, the queue in
> > dbus-daemon could be just replaced by the socket receiving queue in
> > the kernel. We cannot allocate indefinitely there.
> >
> > SOCK_DGRAM Unix sockets has a limit of 10 messages by default. This
> > limit of 10 messages is really low compared to dbus-daemon's
> > max_incoming_bytes 1G. It can be changed
> > with /proc/sys/net/unix/max_dgram_qlen. But it cannot be changed on
> > a per-socket basis.
> >
> 
> Hmm, right. The only solution I see for a kernel implementation would
> be to block. libdbus will have to poll for ready to write and avoid
> blocking writes if the buffers are full or something.
> 
> A low limit (10 or even 100) won't be practical. Basically any limit
> that gets hit in practice is not practical.

If by "blocking" you include returning -EGAIN when the socket is
non-blocking, I would agree :) But:

Consider this scenario with 4 D-Bus peers named A, B, C, D:
- A writes to B
- C writes to D
- D's receiving queue is full.

Whatever happens for C and D (blocking, returning -EGAIN or closing
D's socket), the communication between A and B should not be stopped.

In practice, it is not easy to do: before the kernel receives the
buffer from A, it is not possible to know the recipients because the
message's content need to be read and checked against the match rules.
So when A calls poll() in order to know whether it can send a message
to B, the kernel has no other choice than to check the available space
on receiving queues from all peers on the bus, including D's. poll()
will return "the socket is not ready to write" because of D. Then, the
communication between A and B will be blocked too.

poll() and whether send() can be done without blocking/returning
-EAGAIN need to be coherent. If poll() says "yes, you can send data"
and send() returns -EAGAIN, the process will keep trying in loop, taking
all the CPU.

I don't see any other solution than closing D's connection in order to
save the communication between other peers: "Sorry, you are blocking the
communication of all the peers, I will close your connection!").

Alternatively, we can drop the constraint that a peer cannot block all
the communication on the bus; it can be useful for debugging if
dbus-monitor is slow. It could be an option in session.conf.

Le Tue, 5 Oct 2010 10:36:44 -0400,
Colin Walters <walters at verbum.org> a écrit :

> Though one thing I would note is I might expect the behavior to differ
> for method calls vs signals.  For method calls, one might desire for
> dbus_connection_send() to block (thus restoring reliablity), even
> though it's explicitly documented not to.  So a new method would have
> to be introduced.

In dbus-daemon, the current behaviour differs whether a recipient
receives the message because it is the destination of the message or
because it has a matching match rule on the message (eavesdropping):
- For the destination recipient, the message is dropped and an error
  message is sent to the sender.
- For an eavesdropping recipient, the message is dropped without
  notification.

In general, it is similar to the separation method calls vs signals you
mentioned (match rules are more useful for signals, except debugging
tools like dbus-monitor).

But I don't see how to do that with socket filters in the kernel.

-- 
Alban