To XCB or not to XCB ...

[Jamey Sharp]

On Tue, Aug 29, 2006 at 06:13:33PM +0300, Daniel Stone wrote:
> On Tue, Aug 29, 2006 at 04:39:14PM +0200, Lubos Lunak wrote:
> > Is this about the fact that XCB is split up into a large number of 
> > ridiculously small libraries that'll no doubt make ld.so choke with any 
> > somewhat larger application or is this something else?
> 
> From an embedded (okay, consumer device, whatever) point of view, this
> is an absolute showstopper.  We just can't afford to take that kind of
> hit for every app that links libX11.

libX11 pulls in exactly the same number of libraries with XCB as it did
without, or one more if you want XDM-AUTHORIZATION-1 support. It doesn't
use any of XCB's extension libraries.

Embedded and consumer devices are key targets for not linking libX11 any
more, though. By switching to XCB an embedded developer frees up
probably 700kB of storage.

> > I wonder how XCB should provide "significant" performance
> > improvements over something that's not much more than just a tool
> > for talking the wire X protocol.
> 
> I assume Jamey's talking about how the API's asynchronous, rather than
> libX11's nightmare horror show.

Yes, that's the big one. See the many things we've written about
"latency hiding" or "reply cookies", as well as the "X Window System
Network Performance" paper by Keith Packard and Jim Gettys.
	http://keithp.com/~keithp/talks/usenix2003/

Also, ensuring that developers understand exactly where caching and
other optimization is occuring and can manage it correctly should help.
(I've been told that Qt developers have had trouble with Xlib on that
score, for example.) We hope that better cache locality and such will
help too, though I'm coming to the conclusion that in this space those
effects are pretty small.

On Tue, Aug 29, 2006 at 04:39:14PM +0200, Lubos Lunak wrote:
> On Sunday 27 August 2006 00:04, Jamey Sharp wrote:
> > There is a very small, but measurable, performance hit to using XCB,
> > made marginally worse when Xlib is used on top.
> 
> Is this about the fact that XCB is split up into a large number of 
> ridiculously small libraries that'll no doubt make ld.so choke with any 
> somewhat larger application or is this something else?

I wasn't commenting on that concern, but let's think about it.

According to the evidence from the X Network Performance paper, modern X
apps use about five X extensions. (There's a double-counting error in
the text of the paper, which claims ten extensions, but the data show
between 2 and 7 per app.) Since BIG-REQUESTS is built into libXCB, that
means five XCB libraries per pure-XCB app.

Now compare Xlib. libX11 has BIG-REQUESTS and XKEYBOARD. libXext has
quite a few other extensions, but few applications use LBX, MIT-MISC,
XSecurity, or most of the others in that library. (My server, 7.0.22
from Debian testing, doesn't even have most of those extensions.) Of
extensions used in the data I checked from the above paper, only SYNC
and SHAPE are in libXext. The others each are in their own library, just
like with XCB.

In short, applications like xine that use both SYNC and SHAPE will have
to get them from two libraries instead of one when ported to XCB. The
much more common case of using XKEYBOARD will also require one more
library than before. In essentially every other case, there will be no
difference in the number of loaded libraries. Furthermore, apps that
don't use the 13 extensions in Xext won't be loading the code for them,
and that code need not be stored on disk/flash/ROM if no installed app
uses it.

In all sincerity, two questions: Will a worst-case extra two libraries
be a problem for apps when they've been ported to XCB? And shouldn't we
be fixing ld.so instead of breaking clean library designs?

> > I've been running essentially the current version of Xlib/XCB on my
> > laptop in lieu of stock Xlib since early this year, up to fixing a
> > couple of bugs in that time, and I don't believe there is any
> > perceptible performance difference on real applications,
> 
> Too bad believing happens to be probably the worst tool for measuring 
> performance.

That's absolutely true. So measure it yourself for the workloads you
care about. Qualitatively, the best I can do is point out that I haven't
perceived any performance difference on the workload *I* care about -- a
full-blown KDE or Gnome desktop environment.

Quantitatively, I've done a ton of performance measurement, including
many runs of x11perf. I can tell you from that for most X requests, the
server's processing time swamps request delivery time. For those
requests, XCB gets just as much throughput, or sometimes a little more,
than Xlib. XCB's worst case is the NoOperation request, which of course
is meaningless to measure because it, uh, doesn't do anything. But none
of that will tell you anything about whether XCB will make any
difference to real workloads.

Vincent Torri has reported that a drawing throughput benchmark he was
working with dropped from 133fps to 131fps in the switch to XCB. The
biggest difference was in output queue size: Xlib uses a 16kB queue by
default, while XCB uses a 4kB queue. If you're optimizing for
throughput, a bigger queue helps -- but it's not clear to me that
throughput is the driving force for real X workloads, which is why I
haven't changed the queue size yet. Other issues included that he'd
built XCB without optimization... :-)
	http://lists.freedesktop.org/archives/xcb/2006-May/001568.html

So I don't want to hear more abstract criticism until you really have
something to complain about -- and at that point, we'll do our best to
fix whatever issue you find.

--Jamey
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.x.org/archives/xorg/attachments/20060829/0921cdf9/attachment.pgp>