[systemd-devel] sd-bus: Enabling free-standing, bus-independent plain messages

[Lennart Poettering]

On So, 12.05.19 13:06, Stanislav Angelovič (angelovic.s at gmail.com) wrote:

> Hi (Lennart :)
>
> Quick question: Would it be possible to extend sd-bus to also allow
> creating messages without requiring a bus?

This used to be available but we dropped it, since the serialization
depends on the bus feature set, i.e. wheter fd passing is available,
and whether gvariant serialization is available. Some of that never
materialized IRL due to the demise of kdbus, but the semantics still
make sense: serialization depends on the feature set of the bus, and
hence each message should be associated with a connection.

> Let me explain: Currently, if we want to create any message in sd-bus, we
> need a valid bus pointer. That might make perfect sense for messages that
> come from or will eventually go to bus. But sd-bus also supports plain
> messages. Do we also need that hard msg->bus dependency for plain messages
> which we use only as a local, temporary storage of serialized data?
>
> Some years ago we had a little discussion about a few tweaks in sd-bus API
> to allow modelling the concept of Variant type in higher-level sd-bus
> bindings (
> https://lists.freedesktop.org/archives/systemd-devel/2016-November/037929.html).
> A Variant is essentially implemented as a class around plain sd-bus message
> which is only used a storage of the underlying type. To create a Variant
> instance, we simply call `sd_bus_message_new` factory with type
> _SD_BUS_MESSAGE_TYPE_INVALID. Nice and simple. But that factory requires
> real bus ptr. This leads to a more complicated, less efficient and
> not-that-nice solution that must take hold of some bus (if there is none it
> must create one), must cache the bus (so that it's not created and
> destroyed at every creation/deletion of Variant instance). Yes, we have
> `sd_bus_default_system`, but even with that there is some effort that has
> some pitfalls (e.g. we can't `std::move` a Variant to a different thread,
> because it might potentially outlive the current one). All that
> nomenclature is needed just to get a plain message to read/write
> data from.

So we could readd the ability to create a bus message with a NULL bus,
but I am a bit concerned that people then always pass NULL which might
ultimately result in constant remarshalling if the message is
eventually enqueued on a real bus. When you put together a message it
actually matters which bus connection you do that for...

> Even today, vast majority of these functions doesn't seem to need the bus.
> The exception are `sd_bus_message_seal` and `sd_bus_message_new` itself.
> We'd need to either modify `sd_bus_message_new` to allow taking in NULL bus
> ptr, or creating a new function for creating such plain local messages.
> We'd need to modify `sd_bus_message_seal` to simply consider the fact that
> m->bus can be NULL. And we'd probably need to add asserts for m->bus to
> functions which truly require real bus ptr presence. (We might even provide
> the option to link the message to a real bus later, and introduce new
> `sd_bus_message_set_bus` function, but I'm not sure whether that is good.)
>
> To summarize: sd-bus already offers API for creating plain message. To make
> it fully true, let's just not require the bus there. The proposed change
> would IMHO make  sd-bus message more flexible, and would make it easier,
> more intuitive and more robust to work with and model Variants around such
> free-standing, bus-independent sd-bus messages in higher-level languages.
>
> What do you think?

Note that message behaviour and so on depends in sometimes subtle and sometimes not
so subtle ways on the bus connection used, i.e. whether we are talking
to a real bus or not, and so on. I am not to keen of making this
completely independent I must admit...

What's the usecase for using bus message as general marshalling
storage anyway? Can you elaborate?

A compromise might be that we readd a concept of allowing
bus-independent messages to be generated again (i.e. pass NULL as bus
object when creating a bus message), with the most reduced feature set
possible, and at the same time refuse to enqueue such messages on any
bus, thus forcing people to use sd_bus_message_copy() to explicitly
remarshal for the bus, instead of doing implicitly so. if you want to
prep such a patch I think we should merge it.

Lennart

--
Lennart Poettering, Berlin