dbus-python-in-C and Variants

[Simon McVittie]

A follow-up mail about variants, as promised. I'll answer your questions
first, and describe why I implemented it like this, and some
alternatives, afterwards.

On Mon, 02 Oct 2006 at 18:08:00 -0400, John (J5) Palmieri wrote:
> On Wed, 2006-09-27 at 15:54 +0100, Simon McVittie wrote:
> > * Variants arrive as method parameters, returns from proxy methods, etc.
> >   as Variant objects. This is a bit of a big change, but makes things
> >   completely unambiguous. To unwrap variants like the old Pyrex
> >   implementation did, you can write::
> > 
> >       while isinstance(myparam, dbus.Variant):
> >           myparam = myparam.object
> > 
> >   This should also work (and do nothing) under the Pyrex implementation.
> 
> I'm a little worried about this.  Python is a dynamic language.
> Everything is a variant (it was the purpose of variants in the first
> place).  If signatures are not specified by the service we assume
> variant.  I don't know if strongly typing python here is valuable.  Can
> I use myparam as if it were it's contained type?  For instance if the
> myparam variant held a dbus.Dictionary would print myparam.keys() work?
> More importantly would isinstance(myparam, dict) work?

No, if you want do do that, you need to unwrap the variant, as I
described.

myparam.keys() does not currently work, but could be made to if there is
a compelling reason. I'm a bit reluctant to add too much functionality
though, because eventually I'll have to stop emulating, and a Python
programmer encountering the point where I stopped will continue to be
surprised.

isinstance(myparam, dict) does not return True, and, in any
implementation resembling the current one, never will. Variant is a type
in its own right, subclassing only __builtin__.object.

> > * Variants are now immutable "value objects".
> 
> > * Variants are somewhat more useful: they can be cast using int(),
> >   str(), long(), float(), you can iterate over them with iter() if they
> >   contain an iterable, you can compare them with ``==`` and ``!=``,
> >   and you can hash them if the underlying object supports it.
> 
> Ok, I guess that answers my myparam.keys() issue (or does it?) but the
> isinstance question still remains.

I mean exactly what I said, no more, no less. The iter() change means
that for arrays, structs or dictionaries wrapped in a Variant v,

    for x in v:
        print x

will work. Other methods don't work, unless I specifically write code
for them.

So. Variants, and why I changed the API. I've thought about this quite a
bit and discussed it with the other Collabora people - basically, I'm
aiming for the least astonishing "lossless" implementation.

The API in 0.7 is "lossy" - when inside an exported method implementation
or a signal handler, you can't tell from the Python-level arguments
exactly what went over the wire. Among other things, this makes Matthew
Johnson's binding cross-tests unimplementable - it's impossible to write
a method Identity(v) -> v which does the right thing, because you can't
tell the difference between Variant('abc') and Variant(...(Variant('abc'))...).

To get round this we had the explicitly_pass_message decorator, which in
my view should never be necessary in a high-level binding in a dynamic
language - the kwargs like sender_keyword that have been added are much
better, IMO.

Here are some alternative approaches for dealing with Variants that I've
considered. For each I quote how to code methods VS (which expects a
Variant argument containing the string 'abc', and should put a
Variant containing the string 'abc' on the bus) and VVS (which
expects a Variant containing a Variant containing 'abc', and should put a
Variant containing a Variant containing 'abc' on the bus).

* My current implementation: for D-Bus => Python make everything explicit.
  For Python => D-Bus, when expecting a Variant, accept a Variant as-is,
  but for programmer convenience, if a non-Variant is returned wrap it
  in a Variant automatically.

    @method(in_signature='v', out_signature='v')
    def VS(self, variant):
        assert variant == Variant('abc')
        return 'abc'        # Variant('abc') would also work

    @method(in_signature='v', out_signature='v')
    def VVS(self, variant):
        assert variant == Variant(Variant('abc'))
        return Variant(Variant('abc'))

* The "one level of variant" implementation: for D-Bus => Python
  unwrap exactly one level of "variant'ness", for Python => D-Bus
  (for a variant) add exactly one level of "variant'ness".

    @method(in_signature='v', out_signature='v')
    def VS(self, variant):
        assert variant == 'abc'
        return 'abc'        # Variant('abc') would NOT work

    @method(in_signature='v', out_signature='v')
    def VVS(self, variant):
        assert variant == Variant('abc')
        return Variant('abc')

* The "no variants" implementation: for D-Bus => Python unwrap
  arbitrarily many levels of "variant'ness", for Python => D-Bus
  add one level of "variant'ness", and live with the fact that we
  lose a small amount of type information.

    @method(in_signature='v', out_signature='v')
    def VS(self, variant):
        assert variant == 'abc'
        return 'abc'

    @method(in_signature='v', out_signature='v')
    def VVS(self, variant):
        assert variant == 'abc'     # lossy!
        return Variant('abc')

* The "annotated types" implementation: Throw away the Variant type and
  ensure there is a specialized subclass of a Python type for each D-Bus
  type (so resurrect dbus.Boolean, dbus.String, dbus.Double, dbus.Struct
  as separate types), and give all the D-Bus types an attribute
  dbus_variant_level which is a non-negative integer. This has the issue
  that bool cannot be subtyped, so I'd have to implement dbus.Boolean as a
  subclass of int rather than bool, like in the Pyrex implementation.
  Least astonishment, etc.

  Still, this does end up looking fairly Pythonic in common cases:

    @method(in_signature='v', out_signature='v')
    def VS(self, variant):
        assert isinstance(variant, dbus.String) and variant == 'abc' \
               and variant.dbus_variant_level == 1
        return 'abc'                # auto-wrapped
        # or: return dbus.String('abc') (auto-wrapped)
        # or: return dbus.String('abc', dbus_variant_level=0) (auto-wrapped)
        # or: return dbus.String('abc', dbus_variant_level=1)

    @method(in_signature='v', out_signature='v')
    def VVS(self, variant):
        assert isinstance(variant, dbus.String) and variant == 'abc' \
               and variant.dbus_variant_level == 2
        s = dbus.String('abc', dbus_variant_level=2)
        assert s.dbus_variant_level == 2
        return s

* A variation on the "annotated types" implementation: subtract 1 from
  the variant level when going D-Bus => Python for a variant parameter,
  add 1 to the variant level when going Python => D-Bus.

Thoughts? If you object strongly to the implementation I've gone with so
far, I think the "annotated types" implementation has potential.

Regards,
        Simon

PS: J5, will you be at the GNOME Summit at the weekend? If so,
I look forward to meeting you and arguing about^W^Wdiscussing this in
person :-)