Request for the 1.0 release

[Havoc Pennington]

Hi,

On Sat, 2006-02-25 at 20:15 +0100, Thiago Macieira wrote:
> Hopefully, we'll start porting the DCOP stuff to D-Bus within one month. 
> If we find large issues, it should be in the very beginning.
> 

That sounds like good news. Can we consider you our "point of contact"
keeping track of critical dbus issues? Otherwise it's tough to be sure
which things are considered truly showstoppers by the KDE core devs and
which things maybe would be nice but don't matter too much, especially
if lots of people start to post their opinion. So would appreciate your
ongoing judgment/feedback.

> I don't expect to find many. The most complicated issues should be DCOP 
> code that didn't use the standard interfaces and tools (dcopidl*), 
> instead relying on doing stuff manually. And, of course, code that 
> expects to be able to send any classes just by overloading operator<<. 
> Both cases, though, should be solved in the application, not in the D-Bus 
> protocol, library or binding.

You could probably continue to support operator<< using a struct or byte
array in the dbus message, though of course using it would create
interoperability issues.

What do you think about performance. I'm expecting that dbus is not as
good as DCOP if you just do a "make tons of method calls" benchmark,
though I don't think anyone has done it. I can imagine some people will
consider this critical.

A message prior to the protocol revamp/rewrite:
 http://lists.freedesktop.org/pipermail/dbus/2004-November/001779.html

The protocol revamp (which introduced the "signature string" and
recursive types like structs) basically undid my performance work there
and slowed things down a good bit, though it also perhaps accelerated
the bindings by introducing the signature string and zero-copy array
access. Nobody has done any real profiling or optimization since the
protocol changes. Well, I did just enough profiling to think "this is
kinda hard to make much better without large changes" and left it at
that ;-)

libdbus can be made significantly faster probably, but I don't think my
current libdbus implementation can be made as fast as DCOP or ORBit2.
Essentially I optimized for flexibility, maintainability, and
security/robustness at the expense of some speed. There are some things
we could do like have a "don't validate anything, assume the peer sends
correct protocol packets" mode - orbit and DCOP both basically trust the
incoming data, I believe, and that saves a lot of cycles.

Anyway my preferred approach here is to say "we can always optimize
post-1.0, it's not a showstopper" - I'm just bringing it up because I
expect someone else to.

> So, with any luck, we can say that June should be ok. We don't have it set 
> on stone, but we'll soon have a date when DCOP will be removed from the 
> KDE libraries, thus forcing every application author to update his code. 
> I expect that to be mid-April.

That sounds feasible.

> Given what has been discussed for #1, I expect it to be completed with a 
> few lines of code. In fact, someone has proposed to write a patch, right? 

I think it's probably a day or two of work to get it working robustly
and think about all the race condition cases. But yes I hope someone
will write a patch ;-) if someone volunteered, can you chime in and let
us know if you're still planning to do it?

> It should be a simple app linking to libX11 that gets the information 
> from the root window. This app should probably be installed to 
> ${prefix}/lib/dbus-1.0, since it's not meant to be run by the user. Its 
> path can be hardcoded in the library.

Concur (minor detail for our patch author, use the default root window
for the display, not the root window in DISPLAY)

> As for #2, I do consider it a critical issue because we simply don't know 
> yet how big the impact of supporting those two platforms will be. If we 
> knew it was just a matter of writing the low-level code, I'd be more 
> comfortable.

Agreed that it's unclear at this stage what will be involved. It could
be a long time before someone appears to work on this though.

I would think if nothing else we can address cross-platform issues by
ugly API additions... we can probably make any change we want post-1.0,
it's just a matter of maybe we won't be able to do it in the most
elegant way anymore, we might have to add some grotty or alternative
API.

I can't really imagine protocol changes for win32/mac, beyond new auth
mechanisms ...

> A signal cannot be emitted without the corresponding interface name. If we 
> make it so that methods cannot be called without the corresponding 
> interface name, I fail to see the point in separating interface from 
> method/signal name. Maybe the protocol should merge them in one single 
> header field.

Oh, that's just to avoid having to mess with string parsing. Seems more
convenient. The get_interface()/get_method() accessors can also return a
reference to the header in-place instead of allocating a new copy to get
nul termination.

> And, no, we don't rely on this to work nicely. There's no binding that 
> does because the binding would simply not be working at all.

What I meant was on the caller side, would it uglify any bindings if
they had to provide the interface name. 

Another way to put it, is anybody creating proxy objects that don't know
the interface name of the remote object.

If nobody is then we may as well just require the interface name. It was
optional only because it seemed potentially convenient for app
programmers if they didn't have to type it.

I suppose there's some uninteresting performance concern (sending the
iface name makes the message 20 bytes longer or so on average probably).
Probably it isn't even possible to measure this performance impact
though.

> Yes:
> arguments to dbus_connection_send_preallocated() were incorrect, 
> assertion "dbus_message_get_type (message) != 
> DBUS_MESSAGE_TYPE_METHOD_CALL || (dbus_message_get_interface (message) != 
> NULL && dbus_message_get_member (message) != NULL)" failed in file 
> dbus-connection.c line 2261.
> This is normally a bug in some application using the D-BUS library.
> 

Eh, that assertion is just wrong, given the current spec. If we change
this it will be right though and in fact should be added in a number of
other places. The validation code will also need to reject messages that
are missing the interface.

> It isn't crashing, though. It just drops the message silently (my 
> connection isn't closed).

Yeah, it's a _dbus_return_if_fail(), not a _dbus_assert()

> The spec can mention what the least common denominator is, so that 
> implementors are aware of what is interoperable and what isn't. But, as 
> you said, if the author wants to use non-standard features, that's up to 
> him.

Maybe some of the binding authors could try to chime in here with what
will/won't interoperate with their binding... questionnaire ;-)

 [ ] clients can use parameter-overloaded methods
 [ ] servers can implement parameter-overloaded methods
 [ ] clients can use interface-overloaded methods
 [ ] servers can implement interface-overloaded methods
 Any restrictions on overload details: ______________

> Just please don't ask me to forbid this in the automatic introspection, 
> because it would be an expensive operation. In fact, the automatic 
> introspection is likely to output the same method (same parameters, same 
> annotations, etc.) more than once.

If it's not forbidden the spec should probably say it's allowed and the
details of what bindings should be prepared to handle.

> But without it, I cannot even make blocking calls to myself, because 
> dbus_connection_send_with_reply_and_block queues every message it 
> receives, except for the answer.
> 
> Should the binding instead detect that the call was made to a local name 
> and route the call directly, instead of using a DBusMessage? Or should 
> libdbus do that (listening for all NameAcquired and NameOwnerChanged 
> signals), bypassing the bus daemon?

I'd rather punt it up to the bindings instead of libdbus. The bindings
are also making the overall decisions about when/what-kind of reentrancy
to allow, so are in the best position to do the right thing here.

The Glib bindings already track the NameOwnerChanged stuff in order to
emit destroy signals etc., so could add short-circuiting without new
overhead in theory (though I think the current glib binding decision to
just require async calls in this case is the right one for gnome)

Havoc