dbus-python-in-C initial implementation available

[Simon McVittie]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Sorry for the delay, I've been away from work for a couple of days.

The Variant issue is complex enough that I'm going to discuss it in a
follow-up mail; the rest of your points are addressed below. I think.

On Mon, 02 Oct 2006 at 18:08:00 -0400, John (J5) Palmieri wrote:
> if type(prop) != type({}):
>     raise Exception

It's very rare that this is, in fact, what you want, so it's
unsurprising that this broke. (Incidentally, dbus-python-in-Pyrex did
this a lot in _dbus_bindings...)

> On Wed, 2006-09-27 at 15:54 +0100, Simon McVittie wrote:
> > * Byte is a subtype of str rather than of int, which better matches
> >   Python's conventions for dealing with byte streams. Its constructor
> >   accepts either single-byte strings or integers in the range 0 to 255.
> >   It overrides __int__, but unfortunately int() special-cases strings,
> >   so int(Byte(1)) raises ValueError and int(Byte('1')) gives 1 rather
> >   than the expected ord('1').
> 
> We should find a way around this if we can.  A lot of services like
> Avahi and bluetooth use byte arrays so we should check against them.

Well, the only things I can do are:

- - Byte subclasses str, live with the slightly unexpected int() behaviour.
  This matches the Python socket API, for instance.

- - Byte subclasses int, perhaps give it a str() that's the same as chr().
  This matches the old dbus-python API (sort of, see below) but when
  byte arrays are optionally converted into the ByteArray str subclass,
  having the subscript operator return something that's basically an int
  is a bit surprising.

- - Byte is unrelated to int and str, but has the __int__ and __str__
  methods for coercion.

If int() didn't special-case strings, the first (as I implemented)
would clearly be the best, but as it is, the second might be a better
option. I hadn't realised while implementing just how special the special
case was - it takes precedence over a custom __int__ method!

There is actually code in _dbus_bindings to let the Python programmer
specify how conversions will work - at the moment it's only available
as keyword arguments to the lower-level Message.get_args_list(), but
my plan was always to expose whatever bits of it seem necessary as keyword
arguments to, e.g., the @method and @signal decorators.

I think the amount of this should be minimized (Python follows
"there's one obvious way to do it" rather than TIMTOWTDI) but, for
instance, the ability to get arrays of byte arriving as a ByteArray
rather than a list of Bytes is a valuable capability, I think (if you
want to transfer some sort of avatar image over the D-Bus as an array
of bytes, you don't want individual Python objects for each byte, you
just want a string you can provide to PIL or gtk+).

The current options (all boolean kwargs) are:

- - integer_bytes: if true convert D-Bus bytes to Python ints (loses type
  information, should probably be removed)
- - byte_arrays: if true convert arrays of byte ('ay') into ByteArray
  instead of a list of Byte; lossless, so keep it
- - variant_unpack_level: see discussion of Variant in a separate mail
- - utf8_strings: if true convert D-Bus strings into str objects (always
  UTF-8), if false convert them into unicode objects; lossless, so keep
  it

> > * In method parameters, method returns from proxy methods, etc.,
> >   integers arrive as instances of dbus.Int32 etc., bytes arrive as
> >   Byte, and so on, rather than everything being converted to an
> >   appropriate built-in Python type. This means you can tell exactly
> >   what arguments went over the bus, and their types.
> 
> This is fine as long as we can use isinstance and a base type to check
> them (i.e. they are derived from some basic Python type).

All the IntNN/UIntNN types are derived from either int or long, so the
following code will not raise an assertion failure:

for cls in Int16, UInt16, Int32, UInt32, Int64, UInt64:
    assert issubclass(cls, (int, long))
    assert isinstance(cls(0), (int, long))

In fact, I'll put that in the unit tests right now!

At the moment UInt32 and Int64 are long subclasses even on platforms
where a C long (and hence a Python int) is 64 bits; I might change that,
though.

> > * Array constructor takes arguments (iterable[, signature])
> >   rather than (iterable[, type][, signature]); ditto Variant, Dict
> 
> The one thing I liked about type was that one didn't have to remember
> the D-Bus signature alphabet but type is messy with other constructors
> so I think signature only is the correct choice.  In the end one needs
> to know a little about D-Bus to be a good citizen.

If I do put a type argument back in, I'll probably make it very strict:
I don't think Array([], type=int) should work, as it's somewhat
ambiguous. On the other hand Array([], type=UInt64) is perfectly
reasonable to ask for.

type= does provide "more than one way to do it" though, and isn't really
expressive enough: a{sv}, (aui) and so on are reasonable things to say
in a D-Bus signature, but can't really be expressed in type= without
blurring the distinction between types and instances. The Pyrex
implementation blurs this distinction in ways I'm not convinced are
helpful.

You could, for instance, make Array([], type=(Array([], type=UInt32), Int32))
equivalent to Array([], signature='(aui)'), but I'm not convinced this
is a good idea...

> > * The Boolean, String, Double and Struct types are now aliases for the
> >   built-in bool, unicode, float and tuple types. Their use is vaguely
> >   deprecated.
> 
> I would keep them just for clarity of what the bus supports

I have no intention of removing them, and because they're just aliases
for built-in types, I can't even make them raise DeprecationWarning (nor
would I want to, to be honest). My intention is just to document them like
"Alias for the built-in bool type, provided for compatibility with
dbus-python < 0.80".

> > * dbus_bindings is now called _dbus_bindings, and is considerably
> >   different internally:
> 
> should it not be dbus._dbus_bindings?  Hides it even more I think.

Extensions at the top level are easy to do, extensions inside packages
seem to have some subtleties (dbus_bindings suffered from this!)

I'm a little concerned that if it's dbus._dbus_bindings, importing it
may implicitly import dbus (i.e. dbus/__init__.py) which imports
dbus._dbus_bindings. I'll have to read up on the exact API guarantees of
Python packages to be happy that this will, in fact, work as intended.

> >   * The signature-guessing algorithm used if there is no proper
> >     signature is exposed as a static method,
> >     ``Message.guess_signature(*args)``
> 
> works for me.  No one should use this except internally but there may be
> some debugging uses.

My main use case was "it makes it easier to unit-test", to be honest. I
could rename it to _guess_signature. It's a bit out of place as a static
method really, it should probably be a global function (the
*implementation* should definitely stay in message-append.h though, as
guess_signature() and Message.append() are closely related).

> >   * Some relatively internal methods have been renamed starting with
> >     an underscore - most Python code shouldn't need to use them, and
> >     they expose the full complexity of Messages etc.
> 
> Yes we should expose them through public API only when appropriate 

I might add a PeerConnection at some point which "publicizes" the
Message API for use in peer-to-peer applications. For use with a Bus the
Message stuff should definitely be hidden, though, since the world-view
we want to encourage is the object-oriented idea rather than the
message-bus idea.

> > I've tried to work out who wrote what from the CVS and git history, and
> > put proper copyright notices on things, after noticing that my C code
> > had the first copyright notices in the whole source tree!
> 
> Hehe.  We just took teh D-Bus copyright

Well, that was all very well before the Great Bindings Split... actual
copyright notices per source file are a good move, though!

> > Did Seth Nickell write dbus-python independently, or is it owned by an
> > employer?
> 
> > J5, I assume your bits of it are owned by Red Hat?
> 
> Seth wrote them as an employee of Red Hat so the copyright should be
> attributed to them.  BTW it should be GPL/Academic Free dual licensed
> like D-Bus 

It is. OK, I'll attribute copyright to Red Hat Inc for Seth's contributions
as well as yours. The AFL/GPL combination is permissive enough that we
probably don't hugely care who owns things, but if relicensing is ever
needed or there's some other dispute, we ought to have a comprehensive list.

    Simon
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: OpenPGP key: http://www.pseudorandom.co.uk/2003/contact/ or pgp.net

iD8DBQFFJPhvWSc8zVUw7HYRAtzUAJ4m5bC+QXZ592nyU3PaAmmbLY5ovgCeOI/7
B2jIqkuuRMTz+P7ronpj/88=
=5vPg
-----END PGP SIGNATURE-----