Value and size constraints in DBus Introspection XML

[Simon McVittie]

On Mon, 19 Aug 2019 at 23:57:30 +0000, Matthew Bromley-Barratt wrote:
> Tools like gdbus-codegen will take a file containing introspection xml
> and emit source code that will implements the interface. The comparison
> to other IDLs (GPB, Thrift, Cap'n Proto, XSD, etc) is natural. So I was
> wondering if there were room (an indeed appetite?) for an addition. When
> specifying method arguments, their name, type and direction can be
> defined. My suggestion is, how about value and size constraints too?

I think the important questions to ask about any proposed extension to
D-Bus introspection are:

* Do you expect code generators like gdbus-codegen to generate different
  client-side and/or service-side code as a result?

* Do you expect generic debugging tools like gdbus, qdbus, d-feet, busctl
  to behave differently as a result?

* Are they backwards-compatible, in the sense that adding them to existing
  introspection data will not cause existing consumers to either
  malfunction, or (more subtly) generate code with an API/ABI that is not
  compatible with what they previously generated?

If they are backwards-compatible, then D-Bus introspection already has a
syntax you could use to add this feature: <annotation> elements, with a
name attribute prefixed with a reversed domain name that you control. All
tools are expected to accept and ignore annotations that they do not
understand. This doesn't necessarily require coordination with the D-Bus
Specification, unless the feature is ready to be standardized as a general
D-Bus thing (perhaps in a namespace controlled by the D-Bus maintainers).
However, I would recommend discussing a concrete proposal on this list,
and/or discussing it with the maintainers of major higher-level D-Bus
implementations like GDBus, QtDBus, sd-bus and the various interpreted
language bindings, before declaring it to be stable (and committing
yourself to semantics that might have been a mistake).

If they are not backwards-compatible then I don't think it is appropriate
to add them. Introspection XML is (unfortunately) part of the D-Bus wire
protocol, which has been frozen since 1.0 more than a decade ago.

In general we have used annotations for all non-essential introspection
features (even those invented by the D-Bus Specification,
like org.freedesktop.DBus.Property.EmitsChangedSignal and
org.freedesktop.DBus.Method.NoReply) rather than expanding the DTD to
include entirely new elements which are likely to break existing parsers.

Many D-Bus developers are not particularly happy with the use of
XML for introspection, because fully general XML is a relatively
complicated specification of which introspection only uses a small
subset, and tools maintainers don't necessarily want a dependency on
a full XML parser. Using XML here was an awkward compromise between
human-authorability (for which we would, with hindsight, have been better
off with something less markup-heavy and looking more like pseudocode)
and machine-readability with minimal dependencies (for which the D-Bus
binary wire format is fine - and every D-Bus implementation is required
to understand that *anyway*). We might have been better off if we had
had a rigorously-defined, machine-readable, long-term-compatible encoding
in the D-Bus Specification, a more loosely-defined, human-author-friendly
encoding (with documentation!) used by interface designers, and a tool
that "compiles" the human-friendly encoding into the machine-friendly
encoding.

At various times there have been proposals to deprecate introspection XML
as something that is transmitted and received over D-Bus, and replace it
with a data structure in the D-Bus wire format (using arrays, structs
and dicts), accompanied by a text format that can be transformed
to/from the binary format and is designed to be editor-friendly,
sacrificing efficiency if necessary. If this ever happens, it will have a
representation for annotations as an extension point, but it is unlikely
to have XML-style extensibility.

> For example, an integer argument called "bearing" might from the
> service's point of view have a valid value between 0 and 359, every
> other value being invalid and, if not excluded, potentially leading to
> undesirable behaviour. Whilst we can hand write code to validate the
> value sent by a client, it would be so much nicer if code generators
> could generate this for us. However, they can't do so if the XML doesn't
> contain the constraints information.

Counterpoint: I don't think writing (pseudocode)

    if (bearing < 0 || bearing > 359)
      raise InvalidArgument("Bearing %d is out of range", bearing)

in a method implementation is particularly onerous. This has the advantage
that it makes the intention clear and explicit, and can produce an error
message that takes advantage of domain-specific knowledge that a generic
D-Bus framework would not have.

You would need to write checks like this *anyway* whenever you have a
semantic constraint that cannot be expressed in the IDL, such as an
argument that is constrained by some fact about a different object,
or is constrained by something entirely outside D-Bus (for example
"argument must be a valid POSIX locale", "argument must be the name of
a file on disk" or "argument must be the ifindex of a network interface
on this machine"). In my experience this happens about as often as
simple range-checks.

> Having the constraints expressed in the introspection XML and being
> implemented by code generators would mean that *almost* all aspects
> of the interface are enforced automatically, and recorded in only one
> place (the XML). Thus any generated code in any language gets the same
> rigorous enforcement.

This is only true when every code generator used by implementations of
the interface in question has been changed to emit equivalent checks
before calling into the user-supplied method implementation.

> If the run-time overhead of constraints was not acceptable in released
> code, it would be easy to have the constraints code generation made
> optional. That way development could be helped by having the checking
> in place, but the released version runs quicker without it. Or it could
> be a run-time option, or be performed depending on the uid of the client.

If services will behave incorrectly when given out-of-range parameters,
then they should always check for those out-of-range parameters (either
via generated code or hand-written code). One of the design principles I
follow in designing D-Bus interfaces is "don't be crashable by peers".

    smcv