[systemd-devel] sd-bus: Implementing Variant concept: Copying sd-bus message contents

[Stanislav Angelovič]

On Mon, Nov 28, 2016 at 6:45 PM, Lennart Poettering <lennart at poettering.net>
wrote:

> On Sat, 26.11.16 16:01, Stanislav Angelovič (angelovic.s at gmail.com) wrote:
>
> > Hi,
> >
> > A quick question: Is it possible to append contents (the serialized data)
> > of an sd-bus message to another sd-bus message? Conversely, is it
> possible
> > to extract a part of sd-bus message contents into a separate
> > message?
>
> There's sd_bus_message_copy() which can copy the contents (or a part
> of it) of a message into another message.
>
> There's currently no API that exposes the raw binary bits, and they
> aren't particularly useful anyway on dbus1 as they are not safe
> regarding alignment and you cannot copy them byte-by-byte to arbitrary
> memory locations.


> > Details:
> >
> > In the process of creating a method call message, I would like to do
> this:
> > 1. Create a temporary, empty message A.
> > 2. Serialize data into the message A.
> > 3. Create a method call message B.
> > 4. Serialize some user data into the message B, plus copy the data of the
> > message A into the message B. And send the message B.
>
> sd_bus_message_copy() should work fine for that.


> > Analogously, when deserializing data from the message B upon its
> reception
> > on the server side, I would like to do this:
> > 1. Extract user data from the message B, plus copy selected data into a
> > separate, empty message A.
> > 2. Destroy the message B.
> > 3. In some other context, deserialize real data from the message A.
> >
> > Why am I asking is because we like sd-bus and are building a C++ binding
> on
> > top of it. We stumbled upon a problem when modelling Variant. The typical
> > flow in such a case is that upon e.g. deserialization, first the Variant
> > instances are created from the message contents, and then at some later
> > phase, users want to extract real data from the Variant. In this phase,
> our
> > C++ binding would perform the deserialization from that temporry message
> > (message A) that the Variant would hold.
> >
> > For example, if the method call message carries an array of Variants, we
> > first deserialize the message into std::vector<Variant>, where each
> Variant
> > instance keeps the temporary sd-bus message with the variant data, and
> when
> > the user asks for a concrete data from that Variant, the deserialization
> > from that temporary message takes place.
> >
> > This is the way it's done in e.g. dbus-c++, the C++ binding for libdbus.
> > libdbus API allows creating "empty" messages, and allows copying the data
> > among messages with the possibility of reading from them at the same
> time.
> > Is it possible to create an empty message using sd-bus? Regarding
> copying,
> > we know there's the sd_bus_message_copy function, but once we create the
> > copy message, we cannot read from it later because of the assertion that
> > the message is sealed, which in reality is not.
>
> Hmm, so internally there's bus_message_seal() which I figure we could
> make publically available, so that you can create a new msg, copy the
> data to it, seal it, and read it back.
>

Yes, exactly, I need to create a message copy, but also be able to read
from it afterwards. If invoking bus_message_seal() is sufficient to make
the message readable after creating it, it would be so welcome to have it
in the public API.
Additionally, we'd need a simple factory function for a message that won't
have a context of a method call, a signal or an error message. Just plain
message that will be used for storing some data and later, via
sd_bus_message_copy(), copying the data into a 'real' message. There is
message_new function in sd-bus, but it's internal. Would it be possible to
make it public, just like lidbbus provides dbus_message_new()?


>
> > Is there a way to solve this in sd-bus? Perhaps another way than I
> > described? Many thanks in advance for your response!
>
> There's no nicer way right now. But I think we could add a
> sd_bus_message_dup() or so that duplicates a message, but does so
> efficiently, i.e. shares the memory for the actual payload. This way
> you could duplicate a message to simply get a new read ptr on the
> same, immutable message. I think that would suit your usecase best?
>

sd_bus_message_dup() as you describe it could perfectly be an option, but
-- similarly to sd_bus_message_copy() -- it would have to allow not only
duplication of the entire message, but also a part of it. This could
perhaps be handled -- consistently to sd_bus_message_copy() -- with an
additional 'complete' boolean parameter. The behavior would be the same to
sd_bus_message_copy, but the result would not be a real copy, but just a
view/reference to (if complete==false, then a part of) the original
message. So in my use case example if the message has signature "av", where
the size of the array is 3, then I would like to create 3 messages out of
it, each pointing to the corresponding variant data in the original
message. And -- importantly -- be able to read from those 3 messages
afterwards (the solution to that could be public sd_bus_message_seal(), as
you proposed).
The first phase of reading from the original message would be done by the
C++ binding (creation of 3 messages), the second phase by the user of the
binding (taking data of a concrete type from these messages).

To sum up, we'd need just two simple things:
- be able to create a plain message, a-ka sd_bus_message
*sd_bus_message_new(sd_bus *bus, uint8_t type);
- be able to read back from a message after we've created it (regardless of
whether created by copying from another message, or anew) and appended some
data into it, which now fails on the "sealed==true" expectation.

Support for efficient, shallow copy (sd_bus_message_dup()) would also be
very nice to have. I like this idea.

Thanks a lot for your support!



>
> Lennart
>
> --
> Lennart Poettering, Red Hat
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/systemd-devel/attachments/20161129/9d72c81e/attachment.html>