My monthly 'oh yeah, the platform' mail

[Havoc Pennington]

Hi,

I concur that getting a bundle of the specs to "1.0" state as a unit
would be useful, but I don't see the bright line between a specs release
and a code release. I'll get back to this by the end of the long mail
here.

On technology pimping
===

The "stack" concept is useful but NOT for technology pimping.
A goal of the stack should be to show which technology is *well-cooked*
- and part of the definition of "well-cooked" is "widely de facto shown
to be useful." So it doesn't make sense to use the stack to "push" or
promote technologies, since "already accepted" should be a gating
principle.

Trying to use the stack to push technologies that weren't ready was a
huge bug in GNOME for a while; we now have a pretty large collection of
things that have to be deprecated, and for all of them they were put in
the stack "so people would use them" instead of "because people were
using them." In fact including this stuff in the stack essentially
doomed those libs, because they were not ready to be widely used. The
stack inclusion required ABI-freezing them, and they could no longer be
easily evolved toward something nicer.

Another reason not to go the "push technologies" route is that it can
come back and bite you in the ass. If I can be controversial for a
moment, look at XPrint: it's generally agreed by mainstream toolkit
authors that XPrint is the 100% wrong direction, yet it's being pushed
into the stack anyhow.

So we should not advocate "pushing" technology by getting it into "the
big tarball" unless we are prepared for everyone to play the same game.

Many of the things that have been discussed for the stack are in fact
not ready. Cairo is a good example: most of the desktop world agrees
this is the right path and that it's going well, but it's *not done*.
The API is not ready to lock down. Same for D-BUS.

There are some things that *are* ready. Fontconfig for example is now
used by GTK+, Qt, Mozilla, and OpenOffice.org. The major OS vendors all
default to configuring fontconfig when you configure fonts. This is the
kind of de facto reality that we want to see.

We hope that Cairo and D-BUS and so forth will get there, but they
aren't there yet.


Platform stack is not all-inclusive
===

A couple of clarifications here. "widely de facto shown to be useful"
need not mean 100% agreement. It just means that we can show that a lot
of app authors (or in the freedesktop case, more commonly toolkit
authors) are really going to use the feature. For a spec or stack
element with toolkits as primary customer, a good guideline to think of
might be "at least two out of GTK, Qt, or Swing are going to use the
feature." I don't mean that as a bright-line test just a rough idea of
what we might consider.

This implies that we have a basic goal with freedesktop.org platform of
targeting the "mainstream" open source desktop, rather than some of the
corner cases we could think about. freedesktop.org *the hosting site*
can and does host alternate platforms or projects that don't have this
"mainstream" orientation, but I think most of us talking about this
platform release are thinking about the primary toolkits and
environments.

I would specifically exclude for example environments that take a
radically new or different UI approach - e.g. I'd expect much of
freedesktop.org would not apply to something like Croquet. And I'd also
specifically exclude what I'd call the "classic X" environment (twm,
Xaw, xsm, xdm, etc.) because nobody is actively moving that thing
forward, and most likely they should not be - we should just leave it
alone and keep it working. Churning it up with new spec implementations
would destroy the one value it has: people are used to it already, and
its bugs and features are well-known and reliable.

So guideline 1 for a platform stack: it should be defined to be
"mainstream desktop" and not these other cases. There should be a target
audience.

Guideline 2: it should require that the specs or libraries have de facto
success among the target audience (whether toolkit maintainers or app
developers).

Purpose of the stack
===

Some thoughts on why we'd collect widely-accepted technologies into a
release:
 - snapshot particular branches/versions of these technologies and  
   give them an overall version number; this avoids the issue 
   where environment or app A requires foo 1.0 and bar 1.2 and 
   environment or app B requires foo 2.0 and bar 1.0, and so 
   you can't make things work. Of course if foo and bar 
   support parallel install this doesn't come up... ;-)
   http://ometer.com/parallel.html
 - the other reason to snapshot a set of branches/versions is for 
   testing purposes; we can have everyone testing the same 
   *combination* of platform elements
 - naming the whole big bundle lets something like GNOME say 
   "we require freedesktop.org platform 1.0" rather than 
   "we require the following long-ass list of stuff: ..."
 - OS vendors can also refer to including a particular platform release
 - have a "release train" that people want to catch, which gives some 
   deadlines and incentive to the development process, and a freeze-and-
   stabilize phase as well. This should encourage more 1.0 releases.

These same basic goals apply to a GNOME or X.org or KDE release.

Purpose of the LSB
===

While I see freedesktop.org solving communication, coordination, and
software development problems - it's largely an intra-community effort -
the LSB is solving the ISV problem.

So what the LSB is doing is documenting, testing, and enforcing the
interfaces that ISVs are supposed to use. They collect not only desktop
technologies but also other elements of the Linux platform.

This LSB process *should* be more conservative even than
freedesktop.org. Though they have been moving somewhat too slow on the
desktop front, that's largely due to lack of volunteers and assistance
from what I've seen. Because the LSB goes further than just releasing
the software, and creates validation tests for ABIs (to be sure they
didn't change) and for ISV apps (to be sure they don't have extraneous
dependencies) it simply takes some work. But this stuff is useful.

Here's another twist, though, for the ISV question: freedesktop.org is
largely creating a "backend for toolkits," and only a few of the specs
are really intended to be used directly by applications. So what we'd
expect the LSB to bless is a bit higher-level - GTK+ and Qt, rather than
XSETTINGS or fontconfig.

Another way to look at LSB: they are not releasing software. They're
releasing a contract-enforcement layer between the ISV and the software.
This is a fundamentally different thing from a
freedesktop/X.org/GNOME/KDE release.


ISV Credibility
===

A danger for the open source desktop is that ISVs look at the huge
collection of libraries available in your average distribution, use a
few of the ones that "everyone" knows are crap, and are turned off when
they get shafted by the decision. There's really no good guidance on
which of the zillion things in /usr/lib are considered OK. And for all
of them you can find some nitwit on the Internet who will say it's the
best thing ever.

This is why having a line where we say "these are the libs to use" is
useful. But that line has to be oriented toward *keeping ISVs from
getting shafted* not toward *getting ISVs to use half-baked gee-whiz
stuff*

If we consider this, plus the goal of using a platform release for
stabilization and integration, I hope it's clear that trying to push
technologies before their time is highly counterproductive.

How do we push technologies?
===

So this leaves the issue of how we *do* push technologies. I'm not sure
it's that complicated to be honest. For the most part if you convince
Owen and Matthias Ettrich of a feature for toolkits to use, you have
successfully pushed your technology. Or in the window manager spec case,
you have to convince myself and Lubos. 

For other technologies it's similar; convince the relevant maintainers.

What this involves, usually, is *talking* to those maintainers to get
the right design directions, and being willing to adapt the technology
until it really works for them and addresses the issues. It also
involves solving a genuinely interesting problem, so that the
maintainers care.

If you don't get the right maintainers involved then getting included in
the stack isn't going to make any difference. The software will get
vetoed at the distribution level (I personally find it very enjoyable to
delete misguided nonsense from Fedora and Red Hat Desktop), or just
vetoed because it sits on the disk unused by any app that matters.

X.org
===

An additional question people have brought up is the relationship
between freedesktop.org and X.org. I think potentially X.org could
define the entire platform and eliminate the need for freedesktop.org. 

This hasn't happened because there was resistance to defining the scope
of X.org to be "entire open source desktop" as opposed to "X Window
System." Because the X.org scope doesn't cover freedesktop.org goals we
have a need to do a freedesktop.org platform that is complementary to
the X11 releases.

My expectation is that the freedesktop.org platform would include by
reference the X11 release, or at least those parts that are relevant to
the modern desktop. For example, I don't think twm needs to be defined
as part of the platform that GTK+/Qt/GNOME/KDE rely upon. But
freedesktop.org would say "the modern desktop requires the core X Window
System" and then define additional components such as D-BUS that X.org
feels fall outside the X.org scope.

Perhaps more controversially, I think the same cautions I've suggested
for freedesktop.org platform would apply to X.org releases. So if X.org
is blessing de jure technologies that have flopped de facto, then X.org
has a problem. So far I think X.org is doing mostly fine in this
respect. And since freedesktop.org hasn't done a platform release it's
unknown whether freedesktop.org will do fine.

Anyhow, the point I'm trying to make is that to the extent X.org and
freedesktop.org exist as usefully distinct entities, it's a matter of
technology scope (X11 vs. all of desktop). I don't think there's a
fundamental difference in the kinds of release the two entities are
making, just a difference in what categories of stuff are in the
releases.

There's no real difference between X.org, freedesktop.org, and GNOME or
KDE either.

>From my point of view freedesktop.org just covers the block of
technology in between X.org and GNOME/KDE. If X.org wants to expand up
to meet GNOME/KDE and leave nothing in the middle then that would be
fine.


In summary
===

Getting back to my first paragraph: I don't really see a reason for a
"specs only" release, I think there are some things such as fontconfig
that are ready to be in a platform release that also includes code.

The relevant line to draw is between things that are fully baked and
things that are works in progress. There's *nothing* wrong with works in
progress, in fact as an open source community we're usually too afraid
to try something out and consider dropping it if it fails. However,
there *is* something wrong with considering something ready to bless
before it has some de facto success.

So I like the idea of a platform release. But the idea that it's a way
to "push" technologies is simply broken. It's a way to communicate which
technologies have been successfully pushed *already*.


Hopefully all this makes sense, if I had more time I'd edit and cut the
mail in half, but better verbose than silent ;-)

Havoc