[systemd-devel] sd-bus: Enabling free-standing, bus-independent plain messages

[Stanislav Angelovič]

Hi Lennart,

Sorry for a bit late reply, I've been quite busy recently.

> So we could readd the ability to create a bus message with a NULL bus,
> but I am a bit concerned that people then always pass NULL which might
> ultimately result in constant remarshalling if the message is
> eventually enqueued on a real bus. When you put together a message it
> actually matters which bus connection you do that for...

My proposal is that we would allow passing NULL bus ptr only for
`sd_bus_message_new` function. And such a message would be disallowed from
enqueuing. In functions that need real bus ptr we might place an assert
that bus != NULL. In a limited set of functions, which I mentioned in
previous e-mail (`sd_bus_message_seal` for example), we would allow bus ==
NULL.

> Note that message behaviour and so on depends in sometimes subtle and
sometimes not
> so subtle ways on the bus connection used, i.e. whether we are talking
> to a real bus or not, and so on. I am not to keen of making this
> completely independent I must admit...
>
> What's the usecase for using bus message as general marshalling
> storage anyway? Can you elaborate?

I provided a use case example a few years ago in
https://lists.freedesktop.org/archives/systemd-devel/2016-
November/037929.html, but I'll elaborate more.

sdbus-c++ provides such a generic layer of abstraction that its clients
don't work with messages but call D-Bus methods as-if it was a local
operation, using native types. For that, all D-Bus types have their native
representation. A D-Bus array is represented as std::vector. A D-Bus string
is std::string. And for D-Bus variant there is custom sdbus::Variant class
that needs to be able to store whatever D-Bus types. So for example, for a
method `doSomething` that takes a D-Bus string and an int, and returns an
array of variants, sdbus-c++ generates something like
`std::vector<sdbus::Variant> doSomething(std::string str, int i);`. Clients
call this method, and sdbus-c++ does all the job -- it creates a method
call message, serializes data into it, sends the message, receives the
reply message, and automatically deserializes its contents into
std::vector<sdbus::Variant>. But variant is special. When deserializing,
sdbus-c++ is unaware what real type is under each variant, but it still
must be able to get a given variant out of the message and store it in the
vector to be returned. This is done by `sd_bus_message_copy` of just that
variant message part. It cannot do `sd_bus_message_enter_container` and
`sd_bus_message_read` directly, because sd-bus API requires signature of
underlying variant contents, and this is what sdbus-c++ has no idea about
at this point. It's the client who knows, so the client then at a later
point, in his own code, asks for an object of specific type from that
Variant, like so `double d = variant.get<double>()`. Here is when
`sd_bus_message_enter_container` and `sd_bus_message_read` on variant
message are called, so the deserialization from variant message takes
place. It's something like 2-step deserialization -- but only with
variants, they are special in that.

Of course, our first idea was to use C++ type-erasure technique employed by
e.g. std::any to implement the Variant class. This works fine when we
construct Variant from a native type. But when we need to create Variant
out of a part of incoming D-Bus message, like in the example of method
reply message above, we are stuck. The code would be very complex,
practically impossible to write, due to conversion of run-time combinations
of D-Bus types to static, compile-time type representations (vectors, maps
of various keys and values that could maps again etc...). Hence, using
sd-bus message as an underlying implementation of Variant class is the
simplest and most reasonable solution; we are simply using the concept that
is implemented in sd-bus already. We can easily create Variant out of a
native type (by serializing the value into underlying message), create
Variant out of existing message (by copying the message part into
underlying message), and read data back from Variant (by reading the
underlying message).

> A compromise might be that we readd a concept of allowing
> bus-independent messages to be generated again (i.e. pass NULL as bus
> object when creating a bus message), with the most reduced feature set
> possible, and at the same time refuse to enqueue such messages on any
> bus, thus forcing people to use sd_bus_message_copy() to explicitly
> remarshal for the bus, instead of doing implicitly so.

Fully agree, that is also what I meant. (The reduced feature set would be
the functions I mentioned in the previous e-mail.)

> if you want to prep such a patch I think we should merge it.

OK, I will try. So we don't introduce any new message factory function
without bus parameter, but make use of existing `sd_bus_message_new`,
right? We will modify it to allow NULL bus ptr. However, this function
needs three things from the bus when initializing message:
`message_version`, `can_fds`, and `allow_interactive_authorization`. What
values shall we use there when bus is NULL?

Thank you Lennart!

>
> Lennart
>
> --
> Lennart Poettering, Berlin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/systemd-devel/attachments/20190523/5e55a7ef/attachment.html>