Tree reorganization (Was: Status of xserver/debrix/modular tree?)

[Bernardo Innocenti]

Daniel Stone wrote:

> None of these are subdirectories; they are top-level directories in
> various projects.  If you check out 'xserver' from the xserver
> repository, you get the KDrive-based server; I know debrix could be
> clarified, and indeed, I believe that all the debrix modules are in
> /cvs/xserver/.old, or /cvs/xserver/.junk or something.
> 
> But, in any case, none of these are 'subdirectories' -- they are
> all top-level modules in their own regard, and should not ever be
> checked out.  (Unfortunately, I'm writing this offline, so I can't
> check.)

Yes, that's right.  I have this bad habit of checking out "." or ""
just to make sure I've got everything.

Checking out modules and updating them one at a time
is too boring for projects like Cairo.



>>Modular tree is nice, but I think we should have something
>>similar to GCC's top-level bootstrap to simplify dependency
>>tracking, expecially for distributors.
> 
> I agree, a top-level Makefile would be very nice for people like the
> BSDs who feel it is important to be able to very easily bootstrap the
> modular tree without having to use Python.  But, for most people with
> fully-installed systems, jhbuild, which uses Python, should more than
> do the job, although the module listings will need a bit of updating.

Where are the modules listed in dependency order?  I had to guess
it myself until I got the list from you.

If some people dislike jhbuild, a shell script would still be better
than nothing.  Any book about project management recommends setting
up things so that a single command rebuilds all the system from scratch.

This is invaluable for newbies and saves some time to experienced
developers too.  It's also a prerequisite for running automatic
regression tests.


> Unfortunately 'one command to build them all' also includes FreeType,
> zlib, et al, in the 'all' part, and also random applications to test
> core X functions (if we need to include all these, surely 'renderbench'
> should be included?).

I'd really like to see render_bench in xapps/.  We may even define
the goal of the next Xorg release as "make Render perform as at least
as good as imlib with render_bench".


>>Forks happen continously in the Linux Kernel, and has even some
>>value (remember the VM saga?).  But Linus isn't ever going to
>>allow code implementing the Win32 API in his tree :-)
> 
> Right, but you don't always want to promote them -- there's a
> difference between forking for specific code (e.g. the VM example; X
> could be said to already have this, with the r300, unichrome, GATOS,
> xterm, et al projects), and forking the entire codebase to go in a
> fundamentally different direction.  We're not talking about things
> that can be trivially merged back here, it's about two polar
> opposites.

Yes, the modular tree cannot live side by side with the
monolithic tree.  It would take twice as much resources
to maintain both them once after major changes such
as ANSIfication, XCB integration, etc.

An experimental Xserver along with a stable one is proably
more bearable, but we now have... how many?  Too many to
count.

Several X server flavours have always existed and may be
a desiderable features, but the bulk of the server code
needs to stay together in one place.

A developer who wants to, say, reimplement the server-side
X protocol on top of XCB would have to redo the work for
each live fork.


>>KDE and Gnome (or, better, QT and GTK) can't build on Composite
>>until it's fully hardware accelerated for a large user base.
> 
> Of course they can.  Their compositing managers won't be flawless when
> they first start development, either.  They can build on the current
> sub-optimal codebase, and then be ready with arse-kicking utilisations
> of the current infrastructure, when the infrastructure itself is much
> improved, mainly in terms of speed.

But they can only use the new technology for optional eye candy
like soft shadows and other window-manager related things.

Most of the UI will still run on top of the legacy graphics
pipeline until Render/Glitz/Cairo gets mature and fast.

The GTK guys are considering using Cairo to draw widgets.
QT has this new Andrew API that can use accelerated primitives
of modern APIs such as Quartz and GDI+, perhaps also Cairo.


>>When it's devivered, it will probably take another year or two
>>before users will enjoy it.  MacOSX already had something similar
>>two years ago and Longhorn won't keep slipping schedule forever.
> 
> I believe the Composite code has been working in KDrive for around
> one and a half years now; wasn't it mid-to-late 2003 that it made it
> in?

Yes, but since then things have not improved much, from the perspective
of a user.  Xserver still is a proof of concept that can't be used
on a desktop due to missing drivers and extensions.

I got it to compile with a few trivial changes, but the DRI/GL
enabled server hangs my machine shortly after opening an xterm
window.


>>Basically, yes.  I know it's yet undecided, but a decision
>>should be made in order to ever get it out.
> 
> Sure, but you can't really make a decision without being fully informed,
> and I don't think we can make such a decision until the Mesa-Solo work
> is a lot more mature than it is today.

Thanks to Dave, looks like we're getting to it soon!  I've just
applied his patches and rebuilt, and I'm eager to try it out.


>>I've read the threads... and it seems to me that most (all?)
>>developers agree we will eventually go modular.  What's being
>>debated is whether it will be ye old XAA, KAA or Xglx, right?
> 
> Certainly not all -- there is at least one person who is very strongly
> opposed to modularisation, and others with either mild or reasonably
> strong objections to either the concept or the process that must be done.
> 
> While a majority of the developers would definitely agree with the goals
> of modularisation, not all of them do, and some certainly have very, very
> deeply-held objections to it (like one developer who stated that it
> would never work, and that we would come crawling back to the monolithic
> system a year after modularity).
> 
> Xgl is quite a way off, especially as it is based on the KDrive
> framework, which is simply undeployable as a general-purpose desktop X
> server.  So, right now, the real decision for the Xorg tree is whether
> to stick with the pain of XAA, or rewrite every driver's acceleration
> hooks and go with KAA.
> 
> These are seriously non-trivial decisions.

I understand.  Disagreement on the big pitcure or on details heppens
in all projects.  There won't always be 100% consensus on anything
when multiple developers are involved.

Making hard decisions may take some time, but not as long as it's
taking in Xorg.  Because most thread I've read in this mailing-list
die when everybody have expressed their opinions and are too tired
to keep arguing over and over.

The role of an RM -- or a benevolent dictator such as Linus -- would
be to encourage polite technical discussion, get the facts (TM), and
then make a decision that doesn't make too many people unhappy.



>>That's the kind of issue on which management would make a clear
>>decision.  Making the wrong decision is still better than
>>stalling forever.
> 
> It's not the kind of decision management can necessarily force, though.
> 
> For instance, sunffb didn't even have XAA support for quite a long time,
> despite the obviously established decision to use XAA for acceleration.
> Now, this says more about the sunffb driver than anything, but the point
> is that if you make the wrong decision, not everyone will follow, so you
> could end up with an X server that has one accelerated driver and
> thirty-eight unaccelerated drivers.  Hmm.


The GCC attitute here would be to declare non-conforming modules
obsolete until someone cares enough to bring them up to date.
Maintainship of obsoleted code becomes vacant and anyone can
volunteer.

I've read many many posts asking to keep old stuff in Xorg,
even if it increases the maintenance burden of all developers.
What I've learned from GCC is that patches that remove old
code are more valuable in the long term than patches that
bring in new stuff.


>>>Yes, it would be handy to have.
>>
>>I'd like to contribute this, but I don't know who the
>>maintainers are.  Let's ask the new RM as soon as he
>>gets on the bridge :-)
> 
> 
> drivers/ati: Kevin E. Martin?, Hui Yu (ATI), Michel Dänzer (some
>              components), Ben Herrenschmidt, Roland Bauerschmidt?
> drivers/nv: NVidia (current liason: Andy Ritger)
> drivers/sis: Thomas Winischofer
> fb/: Adam Jackson (de facto)
> hw/xfree86/int10: Egbert Eich
> hw/xfree86/loader: Daniel Stone, Adam Jackson
> 
> et al.

Could we put this list in a MAINTAINERS file in the
root directory or on the Wiki?
 



> KAA can be implemented within Xorg, absolutely.  It's just a lot of work
> that doesn't get you anywhere unless everyone's pulling in the same
> direction.  Telling an entire mailing list of X developers exactly what
> has been done in KDrive is pretty pointless, also.
>
> For various reasons, as I have already stated, KDrive is not, and I don't
> believe ever will be, a general-purpose X server.  It is very good at
> what it is, which is an experimental development platform.

As you said, X carries an incredible amount of legacy
code and should be made lighter to ease maintenance,
provide smaller footprint implementations for embedded
devices, etc.

I've never believed the "let's restart from scratch" way.
Linus showed us that incremental improvements most
often beat total rewrites.

At this time, we already have two competing projects.
I don't know whether it would be easier/better to
import Xorg stuff into Xserver or vice-versa, but
one of these things must be done.

If getting XAA and KAA to work alongside is a lot
of work, maybe layering XAA on top of KAA would be
easier?


>>The only problem is that, work like this would have to be synced
>>up between xorg/xc/ and xlibs/ therefore it's better to delay it
>>until the monolithic tree finally rests in peace.
> 
> Either that, or just work on it in a branch in xlibs, so we have a
> sensible delta: if we have defined deltas, merging suddenly isn't a
> terribly hard problem.
> 
> That said, it's very easy to do it badly; the ANSIfication/^L work in
> xlibs was very disruptive to merging, e.g.

I've read enough about it in that recent thread
to be scared off it completely :-)

-- 
  // Bernardo Innocenti - Develer S.r.l., R&D dept.
\X/  http://www.develer.com/