D-Bus optimizations

[David Zeuthen]

Hi,

On Thu, Mar 8, 2012 at 3:37 AM, Mikkel Kamstrup Erlandsen
<mikkel.kamstrup at canonical.com> wrote:
> I am seeing time and time again people mapping OO designs to bus objects and
> it can only lead to performance problems. They are doing this because this
> is what all online examples tell them to do. One must think in batched
> calls, stateless requests, and idempotency to devise fast and robust
> protocols.

I largely agree that this perceived D-Bus performance "problem" is
that people are doing too many calls and not caching aggressively
enough - you see often see a lot of applications with amnesia, that
is, they request the same info over and over again. FWIW, I don't
think it means that you have to sacrifice OO in your D-Bus API - for
example, things like the org.freedesktop.ObjectManager interface and
org.freedesktop.DBus.Properties::PropertiesChanged signal are two
things that can help _a lot_ in designing an application - as an
example, the udisks2 object model comes to mind (sorry for always
bringning this up as an example)

 http://udisks.freedesktop.org/docs/latest/ref-dbus.html

which greatly helps reduce the amount of D-Bus messages that are
exchanged between the service and clients. Also note that not
everything in that model is a property - for example, ATA SMART
information easily weighs several kB per disk drive and you don't need
that detail very often. That's why it's not a property, you have to
use a D-Bus method to retrieve it, e.g.

 http://udisks.freedesktop.org/docs/latest/gdbus-org.freedesktop.UDisks2.Drive.Ata.html#gdbus-method-org-freedesktop-UDisks2-Drive-Ata.SmartGetAttributes

and this greatly reduces the amount of traffic when loading the entire
object model via ObjectManager.GetManagedObjects() ... (key
information such as Temperature, Failing etc. is available as
properties though).

Also, for bulk data transport, you don't want to use D-Bus - that's why e.g.

 http://udisks.freedesktop.org/docs/latest/gdbus-org.freedesktop.UDisks2.Block.html#gdbus-method-org-freedesktop-UDisks2-Block.OpenForBackup

returns a Unix file descriptor to data transfer instead of some home
cooked D-Bus interface for that.

So, yeah, I think the answer here largely is just getting your D-Bus
API right from the get-go. And, sure, not every problems maps nicely
to an Object Oriented model but some do and for the ones that do, we
have powerful APIs with aggressive caching, resulting in few
roundtrips.

> One point about DBus which I'd very much like to preserve is that there's no
> magic to it. It's really just an old user space daemon written with the
> common tools we all know. This is a great asset that we should not give up
> easily.

Yeah, I really don't see the point in changing or complicating the
setup we have right now.

>  * Determine precisely why the daemon gets choked up. And figure out if some
> of the more down to earth solutions can provide the needed boost. Fx:
>  - Can the daemon or client libs possibly apply some clever pipelining of
> writes/reads to minimize the number of ctx switches?
>  - Optimize message parsing/building?
>  - Would batched Add/RemoveMatches(in as match_rules) calls improve anything
> (that is, if added to the spec)?
>  - Would threading in the daemon help out? Either just some simple
> producer/consumer thing, or a more elaborate work-stealing-threads kinda
> thing[1]?

Also, for the "service is emitting signals but no-one is listening and
dbus-daemon gets woken up to just drop the signal" concern, we can fix
that by having dbus-daemon(1) push match rules out to clients (would
have to do so continuously as other clients do
AddMatch()/RemoveMatch()) so they can drop signals at the client-side
that no-one is actually listening to (this appears to be racy but it's
actually not if you think about it).

Thanks,
David