libdbus or gio dbus for multi-threaded service ?

[Simon McVittie]

tl;dr: Definitely use GDBus if you're multi-threaded.

On Tue, 13 Mar 2018 at 10:03:04 +0000, Daniel P. Berrange wrote:
> Historically since I recall lots of pain around thread safety of libdbus,
> but bugzilla [1] suggests much (all ?) of this has been addressed.

I think all of the known thread-safety bugs have been fixed (at least
on Unix), leaving only the unknown thread-safety bugs (plus a few on
Windows). https://bugs.freedesktop.org/show_bug.cgi?id=102839 was the
most recent.

Honestly, I still wouldn't trust libdbus to be fully thread-safe. Full
thread-safety is hard, and full thread-safety without having a coherent
event loop model is harder. libdbus is "bring your own event loop" so it
can never have a particularly good conceptual model for what runs where,
and in particular it does not have any way to schedule callbacks to be
called in the context (thread) where the user that set up that callback
might expect it to be called.

fd.o#102839 was a bug in code that was already meant to be thread-safe
since at least 2005, and was previously fixed in 2005, 2006, and twice
in 2010. I think the fact that we are still finding bugs in the locking
model in 2018 should tell you something!

GDBus (in GIO) is a lot better designed for multi-threaded use. It
requires the GLib main loop, so it can say "you must iterate each GLib
main context in a thread that holds the relevant lock" and leave it
at that; and it can hand off callbacks to the thread where the library
user expects them to be called.

> Also what is
> the minimum version of libdbus one should consider if using threads ?

1.12.4, and be prepared to require newer 1.12.x branches when you find
and fix further multi-threading issues. The older 1.10 and 1.8 branches
are still supported, but mostly only for security and regression fixes;
they didn't receive patches for fd.o#102839.

> For GIO DBus[2] I'm not finding clear docs about usage with threaded object 
> dispatch.

The general rule for GDBus is that synchronous (blocking) operations
are OK in any thread, async operations are OK in the main thread
and in any thread that has set up its own thread-default main context
(g_main_context_push_thread_default()), and whatever main context is the
thread-default at the time you start an async operation (connect to the
bus, call a method, connect to a signal), that is the same main context
where the corresponding callback will be invoked.

For instance, g_dbus_connection_call() says

    When the operation is finished, callback will be invoked in the
    thread-default main context of the thread you are calling this
    method from

and g_dbus_connection_signal_subscribe() says

    Note that callback will be invoked in the thread-default main context
    of the thread you are calling this method from.

If other documentation is unclear, please contribute GLib patches to
clarify it (since you know where you expected to find it!) via the GNOME
infrastructure.

> Presumably, the implication is that all the DBus protocol I/O 
> will still take place via the main event thread

No, GDBus does all the actual D-Bus I/O in a worker thread behind the
scenes. The worker thread doesn't run user-supplied code, which means all
threads that do run user code (including the main thread) are equal from
GDBus' perspective, forcing the authors of GDBus to get the locking right.

You can see this worker thread in `pstree -t` output, as the `gdbus`
thread. virt-manager is an example of a program in the libvirt family
that has a gdbus thread (or at least, it does on my laptop).

> Presumably [...] something like GTask
> would have to be used to execute object dispatch asynchronously by spawning
> threads on demand for operations with long execution time.

I think GAsyncResult is a good interface for async operations (known as
"promises" in some non-GLib ecosystems) and GTask is a very convenient
way to implement or work with async results, but in my experience it's
relatively rare that you will need to use g_task_run_in_thread() or
g_task_run_in_thread_sync() with GDBus, because all the time-consuming
GDBus APIs like "connect to bus" and "call a method" have an asynchronous
version anyway - so you can mostly just work with GAsyncResult and
callbacks/"promises" in your main thread, in the same way you would for
networking or libdbus. You start a time-consuming operation and go back
to your main loop, and some seconds or minutes later, a callback that
gives you the result is scheduled to be run by that main loop.

g_task_run_in_thread() is mostly there as a way to take a synchronous
operation like file I/O and make it look like an async operation. For
APis that are already asynchronous (like networking and D-Bus) you don't
usually need that.

    smcv