IDL language

[Havoc Pennington]

Hi,

On Fri, May 8, 2009 at 11:59 AM, David Zeuthen <david at fubar.dk> wrote:
> Sure, they are mostly useless. But what about people who wants to
> consume your service from, say, C or C++?

struct names can be in the _IDL_ (which may have corba-idl-ish syntax
or xml syntax or binary syntax, irrelevant for this point). They don't
belong on the wire, in the protocol. Object types aren't in the
protocol either. dbus enforces types in the binding; not on the wire.
The wire types are only the primitive types - the types that are
needed to describe the serialization.

I had elaborated on this in much more length in the gvariant thread:
http://mail.gnome.org/archives/gtk-devel-list/2009-April/msg00078.html
http://mail.gnome.org/archives/gtk-devel-list/2009-April/msg00083.html

If we start deviating from this design decision, then the "dbus stack"
will rapidly be an incoherent mess, whether you would have chosen to
go the other way or not, this design decision has a lot of ripple
effects and we need to be consistent and coherent on it.

> and GetAtaSmart() has a out_signature of (sa{sv}) where the first string
> has the value "dynamic-struct:org.freedesktop.DeviceKit.Disks.AtaSmart"
> and the dict has the strings "time_collected", "passed", "passed_valid"
> and "attributes".

Horrible hack, imo, that has no business existing. If you're sending
types over the wire like this, then something in the way you're doing
things is _wrong_, because both sides of the conversation should
_already know the type_ ... they both need to know the interface
contract to begin with or you couldn't have written the code! So why
put this on the wire? It can only be due to some sort of design bug.

OK, there's one exception: if you put this in a variant, i.e. you have
an interface contract that involves RTTI.

I guess my basic view is that making all of dbus an uglified complex
type-repository-requiring hell in the mold of CORBA, just to support
using structs in variants for generated-code bindings, is a bad
tradeoff. ;-) Because the _only_ time you need this type tag is when
you want to bind to a generated struct type (rather than some sort of
generic hash table or tuple), and you have a variant so the interface
contract does not already specify the type.

So I'd punt here. C++ programmers can use a map<string,variant>
instead of a struct AtaSmart, when said hash appears as the value of a
variant. In return, we can save thousands of lines of code and
complexity.

But, short of values in variants, I think you get what you want
without putting this in the wire protocol. You just need it in the
_IDL_, not on the wire. And you can still have statically-declared
struct types in generated code.

> This is a nice construct because for non-private services you want to be
> able to add members to structs without changing the on-wire ABI.
> Everyone does this (I bet you even do things like this yourself in your
> litl APIs) and having this codified is thus a Good Thing(tm).

It just does not come up with JavaScript - it's a complete non-issue,
because there aren't any static types.

The caller side is like:

 proxy.FooRemote({ key1: value1, key2: value2 });

the callee is like:

 Foo : function(myDict) {
   whatever(myDict.key1, myDict.key2);
 }

That's all you have to do. If key1 or key 2 is missing, it will throw
an exception eventually, will be caught by unit tests or break.

There's no true static typing once IPC is involved anyway. I mean, you
can have generated typechecks for what goes over the wire, but if the
types are wrong at runtime, you're going to get a runtime error. IPC
always means runtime errors are possible.

> For example, if I want to use the DeviceKit-disks API from python or
> Javascript (and I do), it's *a lot* easier if, hey, I can actually read
> some docs explaining this API. Heck, I could even generate proxies if I
> wanted to. That would give me completion and other crap if I used a
> sufficiently advanced IDE for dealing with this.

Sure, but that's not relevant to what I'm saying. You can have docs
and type annotations and what have you, in the IDL (which can be XML
syntax or some other syntax). But it doesn't belong on the wire.

>> The litl codebase, for example, has loads and loads of dbus that is
>> completely internal, and completely JavaScript-to-JavaScript. Having
>> to deal with IDL for that is just silly.
>
> This sounds good until you hand over to maintenance of that codebase
> over to some team in Elbonia [1]. Or if you decided to open source it.

I don't mean "internal to litl the company" I mean "internal to the codebase"

It's just an app made up of multiple processes.

I don't have an IDL file describing the API between every file in my
app. If some files happen to be joined by a wire protocol instead of
an in-process calling convention, big deal; I still don't want an
extra IDL file. It's JavaScript on both sides and always will be. You
can read the interface to the JS just as well whether it's an
in-process or out-of-process object.

IDL-ish stuff is for defined "published" interfaces with multiple
implementations. We do have an "IDL" like concept in gjs, but it's
used not only for dbus, but also sometimes for in-process interfaces
such as plugin interfaces, with lots of implementors. Whether to
"declare and document" an interface is totally separate from whether
to marshal it over the wire or not. Two different decisions.

In gjs, these things are orthogonal. You can define an interface (and
validate that an object implements it). You can do this for an object
for dbus export, or one that's intended to be purely internal. And
then you can export an object over dbus, or not. You can export an
object whether it formally defines its interface or not.

> I'd argued above that a higher-level syntax is needed unless you want to
> reverse engineer what different instances of a{sv} means. Stuffing that
> stuff into D-Bus introspection XML seems incredibly backwards and
> pointless to me. Just make your gobject-introspection scanner output IDL
> in the first place.

IDL syntax has one really huge downside vs. XML: most languages do not
have an IDL parser.

libIDL is a painful library to use; even if it were not, having to use
some special library is a big pain.

Everyone already has an XML parser; everyone already knows how to use
an XML parser.

Also, all the dbus bindings already support the particular XML format
we're using.

But again, look: nobody is stopping you from doing an IDL syntax, in
fact it was in the original plans many years ago, and is one reason
the old dbus-glib binding tool thing was going to have multiple
input/output formats.

All you do is write an idl2xml thing; and then in your source tree,
you just have IDL; and you have a little makefile rule to build the
XML and install.

> Also, do not underestimate that it's really nice to just be able to do
>
>  $ less /usr/share/dbus-1/idl/org.freedesktop.DeviceKit.Disks.didl

$ dbus-xml-to-idl
/usr/share/dbus-1/idl/org.freedesktop.DeviceKit.Disks.xml | less

is not really that bad.

Havoc