How to ensure DBUS is broken (in clusters and bus-connected NUMA machines)

[Simon McVittie]

On 02/10/15 01:47, Linda A. Walsh wrote:
> re: #4 -- imagine a compute engine that requires expensive authentication
> and/or link encryption over a cluster internal bus  -- it would be
> a design killer.

If this is "a single machine" in the sense that it runs one kernel, then
D-Bus should be using AF_UNIX sockets, as it does by default on all Unix
platforms. Those are basically memory being copied between processes, so
they might not be optimally fast in a NUMA architecture, but that's the
price you pay for having a more complicated memory architecture.

(If AF_UNIX is too slow, then you probably want something with shared
memory where the endpoints implicitly trust each other. D-Bus is not
designed to be that protocol; one of its design assumptions is that
endpoints may be mutually distrusting.)

If it's a cluster of machines each with their own kernel, then D-Bus is
not designed for this, and I would recommend using a different protocol
with different design trade-offs. Quoting D-Bus' specification:

"""
 Uses of D-Bus include notification of system changes (notification of
when a camera is plugged in to a computer, or a new version of some
software has been installed), or desktop interoperability, for example a
file monitoring service or a configuration service.

 D-Bus is designed for two specific use cases:

* A "system bus" for notifications from the system to user sessions, and
to allow the system to request input from user sessions.

* A "session bus" used to implement desktop environments such as GNOME
and KDE.

 D-Bus is not intended to be a generic IPC system for any possible
application, and intentionally omits many features found in other IPC
systems for this reason.

 At the same time, the bus daemons offer a number of features not found
in other IPC systems, such as single-owner "bus names" (similar to X
selections), on-demand startup of services, and security policies. In
many ways, these features are the primary motivation for developing
D-Bus; other systems would have sufficed if IPC were the only goal.
"""

> You really DON'T want encryption in such cases, as no encryption layers
> that I know that can even be used on a 1Gb connection without a notice-able
> impact in speed and latency.

Neither the D-Bus protocol nor the reference implementation is designed
to saturate gigabit links. If it happens to be able to, great (although
I'd be quite surprised).

D-Bus is designed to be a "signalling" protocol, not a "bulk data"
protocol: messages should travel when something "interesting" happens,
not continuously. If your use-case involves a continuous stream of data
between nodes, there are plenty of other options.

-- 
Simon McVittie
Collabora Ltd. <http://www.collabora.com/>