D-Bus Versus Varlink

[Thiago Macieira]

On Tuesday 2 April 2024 07:35:19 PDT Lennart Poettering wrote:
> > This basically showed that CBOR is better in almost all benchmarks. So why
> > use JSON if you can use CBOR?
> 
> Because it makes things *so* *much* *more* *debuggable*.
> 
> If I use "strace" or some other debugging tool on a complex systemd
> session i can suddenly understand immediately what is going on,
> because I can directly make sense of every method call and every
> method reply becaus JSON is readable as tetx.

That depends on who's doing the debugging. I don't deny that it's harder to 
read. If I strace the systemd-resolved transmission, I can immediately tell 
what it is:

recvfrom(3, "{\"parameters\":{\"addresses\":[{\"ifindex\":4,\"family\":10,
\"address\":[32,1,72,96,72,96,0,0,0,0,0,0,0,0,136,68]},{\"ifindex\":4,
\"family\":10,\"address\":[32,1,72,96,72,96,0,0,0,0,0,0,0,0,136,136]}],
\"name\":\"dns.google\",\"flags\":1048577}}\0", 135152, MSG_DONTWAIT, NULL, 
NULL) = 224

If it were binary, I might not know what it is.

But for anyone who's deeply involved with the project in question, they can 
decode CBOR or D-Bus binary in their heads (I know I can).

> Sure, CBOR would make things a bit faster, but only a a couple of 10%
> or so (see zeeshan's numbers). That's not something I am too much
> concerned with. The debuggability is *a* *lot* *more* *important*.

I've posted why I am not completely convinced by those numbers. That's also 
ignoring some other advantages, like the ability to use small integers for map 
keys instead of (repeated) strings. Of the 224 bytes of the payload above, 64 
are the string keys. The 16 bytes of the IPv6 were transmitted in 39 bytes.

In the other email, you listed as a advantages:
> * Suitable to streaming large amount of data (i.e. useful for listing
>   larger directories of things, where the dbus broker would normally
>   kick you off the bus)
> * Defined behaviour for protocol upgrades (i.e. start with Varlink,
>   then clearly switch over to downloading blobs without varlink, on
>   the same connection)

With CBOR, you could do both without leaving the protocol and with minimal 
overhead, because it *can* transmit binary, unlike JSON.

But I will agree with you that, given proper parsers, the cost of the context-
switch and other inefficiencies will dwarf the parsing.

> (Also, JSON is what the web community generally uses, and Varlink is
> supposed to be easy, and natural to use via an HTTP proxy, i.e. it's
> request/response based, so if you glue a GET in fron of each method
> call and an HTTP 200 OK after each method reply you are basically
> already there.)

These days, one expects to start an SSL/TLS session, even for same-network 
services, so if you're using a proxy, you'll use CONNECT.

On the other hand, if you're expecting to talk performance and minimal 
resources, it would be the IoT community, not the web one, to talk to. The 
IETF CoRE WG created CoAP, which is very similar to HTTP in semantics, but 
simpler (think of it as an evolution of Google's QUIC, the same as CBOR being 
an evolution of MsgPack). And there you'd have UDP and DTLS.

> Round trips kill perfomance. Marshalling is much less
> expensive. Context switches are what today's CPUs are still shit
> at. Parsing streams of bytes is what they are really efficient at.
> hence, if you care about performance, look at the roundtrips – the
> marshalling is just a minor factor.

-- 
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
   Principal Engineer - Intel DCAI Cloud Engineering