Status of xserver/debrix/modular tree?

[Daniel Stone]

On Sat, Feb 12, 2005 at 05:13:32AM +0100, Bernardo Innocenti wrote:
> Daniel Stone wrote:
> >On Fri, Feb 11, 2005 at 06:47:32AM +0100, Bernardo Innocenti wrote:
> >>Thank you.  How can I get a list of all... er... repositories?
> >>I have very little experience with arch/bazaar and, no I have
> >>not RTFM yet.
> >
> >I think it's 'baz categories'.
> 
> Hmm... it just prints the list of modules I've already checked out.
> Or are there just these?  There used to be more pieces back when
> Debrix was in CVS.

That's all I've tagged back in -- I've been stupidly busy lately, so
the time I planned to take to work on Debrix has been totally swallowed
up. :(

> >>I used to find it very useful to checkout *everything* in a
> >>single command...
> >
> >Yeah, it's drm/libdrm in the DRI repository.
> 
> I realized this later, thanks.
> 
> Back when freedesktop.org was started, I very much appreciated
> the online build instructions for the xserver project.
> 
> For xlibs and xapps, a top-level configure script to build
> all subprojects in the correct order would be even more useful
> than up to date documentation.
> 
> That would be an easy thing I could try to contribute.

Well, this is the bootstrap order:
xtrans Xproto XCBProto Xau Xdmcp XExtensions XCB XCalibrateExt FixesExt
ResourceExt Randr Render X11 DamageExt CompositeExt ICE Xext Xfixes
Xdamage Xcomposite Xrender Xrandr Xi Xv SM Xt PanoramiXExt Xinerama Xmu
XRes Xcursor Xpm Xaw ScrnSaverExt Xss xkbfile xkbui RecordExt Xtst XTrap
XF86DGAExt Xxf86dga XF86MiscExt Xxf86misc XF86VMExt Xxf86vm XCalibrate
DMXExt Xdmx EvIEExt Xevie Xft Xfont FS

Then libdrm, then debrix, then debrix-input-{keyboard,mouse}, then
whichever debrix-driver-* module you desire.

> >>That's where a designated (RM|board|dictator|whatever) would
> >>step in and make a decision.  Even the wrong decision is better
> >>than such a limbo of uncertainty.
> >
> >And, as I said, we are in the process of appointing a release manager
> >for 7.0.
> 
> I see.  As soon as he's in charge, I'll ask him if we can get
> a release plan and some guidelines.

What specifically do you want to find out that's not already avaialble,
given the general consensus that 7.0 will be a fully modular release?

> >>The Mesa project made a good move by importing the GLX stuff
> >>from Xorg, but the files are still present in the xorg tree
> >>and are soon going to get out of sync.
> >
> >Right now, it is not entirely possible to build Mesa out-of-tree and
> >still get all the features you get with an in-tree build.
> 
> Since a few months, I already build Mesa standalone and use it
> with the regular Xorg server.  GLX and DRI both work fine and
> I get twice glxgears fps than with the 6.8.1 libGL.
> 
> What am I missing?

Maybe it already works, but my understanding that even accelerated
direct was not possible if you built the monolithic tree without
any of the Mesa bits, and then built Mesa out-of-tree.

> >>All repositories are full of old junk such as:
> >>
> >>fdo/dri/drm-old
> >>fdo/dri/xc
> >>fdo/mesa/Mesa-oldtree
> >>fdo/xorg/debrix
> >>fdo/xorg/testdir
> >>fdo/xserver/debrix*
> >>etc.
> >
> >So?  These are there for historical purposes.  You don't have to check
> >them out if you don't want to, and there's no sense in throwing history
> >away.  In fact, I would say that it's a positive step, given that it is
> >showing an obvious deprecation of code.
> 
> I'm not arguing we should "rm -rf" them from the repository,
> just "cvs remove" them so the old stuff does not get in the
> way when you grep through the repository.  One can always
> checkout an old revision in case it's needed.
> 
> Not checking them out is an option, but it's harder for
> subdirectories.

None of these are subdirectories; they are top-level directories in
various projects.  If you check out 'xserver' from the xserver
repository, you get the KDrive-based server; I know debrix could be
clarified, and indeed, I believe that all the debrix modules are in
/cvs/xserver/.old, or /cvs/xserver/.junk or something.

But, in any case, none of these are 'subdirectories' -- they are
all top-level modules in their own regard, and should not ever be
checked out.  (Unfortunately, I'm writing this offline, so I can't
check.)

> >>Not to mention imported copies in the extras/ subdirectory
> >>of xorg/xc/ which are mostly a relic of XFree86.
> >
> >Not really.  Apparently some people actually do want to build FreeType,
> >zlib, xterm and all that sort of stuff, and x86emu, for one, thanks to
> >not being a shared library, needs in-tree building.
> 
> It's the same bootstrap problem we mentioned earlier: despite
> imake, building everything in xorg/ is much easier because it
> follows the "one command to build them all" rule.
> 
> Modular tree is nice, but I think we should have something
> similar to GCC's top-level bootstrap to simplify dependency
> tracking, expecially for distributors.

I agree, a top-level Makefile would be very nice for people like the
BSDs who feel it is important to be able to very easily bootstrap the
modular tree without having to use Python.  But, for most people with
fully-installed systems, jhbuild, which uses Python, should more than
do the job, although the module listings will need a bit of updating.

Unfortunately 'one command to build them all' also includes FreeType,
zlib, et al, in the 'all' part, and also random applications to test
core X functions (if we need to include all these, surely
'renderbench' should be included?).

> >Yes, I've not had time to update it lately.  Since I was last working
> >intensively on Debrix, I've joined a company which has built itself and
> >released a distribution in the space of a few months, and spent
> >virtually half a year overseas.  I've really just not had the time to
> >bring everything back up to date.  Indeed, my mail is still on a
> >'temporary arrangement' that I brought up in November, and intended to
> >last about two weeks.
> 
> I did not mean to criticize your work, sorry.  I just think
> that more people would help (at least by reporting bugs) if
> it was more visible.

It needs visibility, but the codebase is currently embarassingly old,
and needs to be updated to the current monolithic-tree codebase.  I was
hoping to do this last week or something, but work and life got quite
badly in the way.

> Daniel Berlin finally convinced everyone that migrating to SVN
> will solve most of the problems we face today with minimal
> changes in the current workflow and infrastructure.
> 
> Of course Subversion is no way near a real distributed VCS, but
> it's quite mature and better than CVS in many ways and people
> don't need to learn how to use it.
> 
> (this is not svn propaganda, I don't use svn myself and don't
> have an opinion on arch VS. svn or similar issues).

Personally, I'm not convinced by SVN, since its model doesn't solve
a lot of the fundamental problems we have with CVS, IMO.  I think
the system with the most compelling foundations and the most promise
is Arch, which is a great system with a rather horrid UI as it stands
(albeit, one that's rapidly improving).

As much as CVS sucks and is an utter pile of crap, I don't think any
of the replacements are yet compelling enough to look at; especially
ones which do not solve its fundamental problems.  If we move now, I
can guarantee you that we'll still be on that system in five years,
so it had better be a damn good one -- never underestimate the inertia
once you have moved to a new system.

I think that in a year, we'll be able to sit down and have a more
realistic discussion about systems such as Bazaar (XDC 2006?), but
right now, none of the alternatives to CVS bear considering IMO.

Realistically, the only choice for us is to continue hobbling along
with CVS, and just deal with the fact that it sucks.

Bias disclosure: I work for Canonical, who produce Bazaar, which is
a friendly fork of tla to improve the UI.  But I don't work on Bazaar
itself.

> >>People are intelligent enough to see that someone must be leading
> >>the project... and in the best interest of the project one should
> >>either follow his decisions, try to change his mind, or leave.
> >>
> >>Any society works this way, except the Borg :-)
> >
> >So, in that case, you are effectively forcing them to leave.
> 
> The "leave" part in my statement could be rephrased as "work alone
> on a fork".  It doesn't mean dissidents are banned forever.
> 
> Forks happen continously in the Linux Kernel, and has even some
> value (remember the VM saga?).  But Linus isn't ever going to
> allow code implementing the Win32 API in his tree :-)

Right, but you don't always want to promote them -- there's a
difference between forking for specific code (e.g. the VM example; X
could be said to already have this, with the r300, unichrome, GATOS,
xterm, et al projects), and forking the entire codebase to go in a
fundamentally different direction.  We're not talking about things
that can be trivially merged back here, it's about two polar
opposites.

> >X.Org has some hard decisions ahead that need to be made, and rushing
> >them wielding a big sword of authority would be doing everyone a
> >disservice.
> 
> A good RM would first try to get consensus and not rush
> things beyond what can be achieved in practice.

Yes.

> GCC and Mozilla both learned to make realistic feature
> plans and release schedules.
> 
> Xorg seems to be able to perform frequent releases too,
> which is good, but features too often don't match original
> plans.

Again, a lot of this has to do with the fact that we're woefully
undermanned, so the loss of one contributor can totally cripple an
entire portion of the project.  If we lost Roland Mainz, Xprint would
fall apart.  If we lost Egbert Eich, I get the feeling int10 would
disintegrate.  The nVidia driver is done entirely by Mark Vojkovitch,
with Alan Coopersmith constantly playing with merges there.  The
Radeon display detection stuff depends almost entirely on Ben
Herrenschmidt.  And so on, and so forth.

The problem is that if one person disappears or gets unavoidably busy,
then entire chunks of the project can just fall off in terms of
development.  Witness the current situation with debrix, where the code
is lagging badly behind.

> >The Radeon driver has Render acceleration, but yes, while XAA remains,
> >it is arse-slow, and it sucks.  But the entire Composite framework is
> >there and working.
> 
> KDE and Gnome (or, better, QT and GTK) can't build on Composite
> until it's fully hardware accelerated for a large user base.

Of course they can.  Their compositing managers won't be flawless when
they first start development, either.  They can build on the current
sub-optimal codebase, and then be ready with arse-kicking utilisations
of the current infrastructure, when the infrastructure itself is much
improved, mainly in terms of speed.

> When it's devivered, it will probably take another year or two
> before users will enjoy it.  MacOSX already had something similar
> two years ago and Longhorn won't keep slipping schedule forever.

I believe the Composite code has been working in KDrive for around
one and a half years now; wasn't it mid-to-late 2003 that it made it
in?

> >The original question of your thread is which acceleration theory should
> >we go for in the long term?
> 
> Basically, yes.  I know it's yet undecided, but a decision
> should be made in order to ever get it out.

Sure, but you can't really make a decision without being fully informed,
and I don't think we can make such a decision until the Mesa-Solo work
is a lot more mature than it is today.

> >No, beacuse this is a massive and ongoing debate with very, very
> >strongly-held feelings on both sides.  It's not a simple problem to
> >solve.
> 
> I've read the threads... and it seems to me that most (all?)
> developers agree we will eventually go modular.  What's being
> debated is whether it will be ye old XAA, KAA or Xglx, right?

Certainly not all -- there is at least one person who is very strongly
opposed to modularisation, and others with either mild or reasonably
strong objections to either the concept or the process that must be done.

While a majority of the developers would definitely agree with the goals
of modularisation, not all of them do, and some certainly have very, very
deeply-held objections to it (like one developer who stated that it
would never work, and that we would come crawling back to the monolithic
system a year after modularity).

Xgl is quite a way off, especially as it is based on the KDrive
framework, which is simply undeployable as a general-purpose desktop X
server.  So, right now, the real decision for the Xorg tree is whether
to stick with the pain of XAA, or rewrite every driver's acceleration
hooks and go with KAA.

These are seriously non-trivial decisions.

> That's the kind of issue on which management would make a clear
> decision.  Making the wrong decision is still better than
> stalling forever.

It's not the kind of decision management can necessarily force, though.

For instance, sunffb didn't even have XAA support for quite a long time,
despite the obviously established decision to use XAA for acceleration.
Now, this says more about the sunffb driver than anything, but the point
is that if you make the wrong decision, not everyone will follow, so you
could end up with an X server that has one accelerated driver and
thirty-eight unaccelerated drivers.  Hmm.

> >>Kdrive already supports all required Render acceleration, at least for
> >>ATI.  Xgl would even make all the accelerated 2D drivers pretty useless
> >>by relying only on DRM+DRI+Mesa.  Am I misinformed?
> >
> >Yes, but KDrive uses an entirely different acceleration architecture and
> >massively different systems for managing, well, everything.  It's not even
> >close to the same DDX as Xorg.
> 
> Rewriting all 2D drivers almost from scratch seems too
> expensive to me...

Yes.

> Distributions would fall into the same hell that happened
> when XFree86 4.0 came out without drivers for many common
> chipsets and XFree86 3.3.x remained around for years.

Yes.  Which is just one of the many reasons why moving to KDrive is
absolutely staggeringly impractical.

> >If you used Xgl, then you could accelerate all your 2D operations through
> >OpenGL by reworking them into 3D primitives.  But today's current Xorg
> >drivers would be left by the wayside, yes.
> 
> Running a full-GL desktop would be very sexy, but there's still too much
> popular hardware that don't support (fast enough) 3D acceleration.
> Linux distributors wouldn't take this route, at least not for a
> few years.

Yes.  (For what it's worth, my day job is maintaining X for a reasonably
large new distribution, and I previously co-maintained X in Debian.)

This sort of thing is why these sorts of decisions are seriously
non-trivial, and why having someone stand up the front waving his hands
and saying, 'I want GL!' is pretty pointless.  You need everyone on board
for this sort of staggering change, and you need lots of time to discuss
and fully bang out your architecture.  Else you're going to end up with
unmititgated disaster, and eventually someone will get pissed off enough
with the situation to go fork the codebase from some more sensible point
than what it is and make it usable for everyone.  At which point, you
have just lost.

> >These are major shakeups being proposed, both of them.  And I think that
> >moving to KAA would be very worthwhile, but a huge amount of effort to
> >migrate to.
> 
> Isn't there a way to fix XAA incrementally?  I don't know that
> code well enough, but even Microsoft could retain existing
> video drivers while adding fast alpha blended primitives to GDI.

I don't believe so, no.

> >Ehm, it also introduced the Composite extension to the Xorg DDX (and
> >was the first Xorg to ship with Damage and Fixes enabled by default),
> >and much development such as that on XEvIE happened, as well as a few
> >major changes within the X server.
> 
> Yeah, I know... and it's impressive work too, but not in
> the direction that was originally envised.
> 
> That's what I meant when I first said "continously shifting
> focus".  Extending the old Xorg server does not bring
> Xgl/KAA/whatever any closer and increases the amount of
> maintenance work required to keep the other servers
> synchronized.

KAA can be implemented within Xorg, absolutely.  It's just a lot of work
that doesn't get you anywhere unless everyone's pulling in the same
direction.  Telling an entire mailing list of X developers exactly what
has been done in KDrive is pretty pointless, also.

For various reasons, as I have already stated, KDrive is not, and I don't
believe ever will be, a general-purpose X server.  It is very good at
what it is, which is an experimental development platform.

> >>Ok, but wouldn't a MAINTAINERS file in Xorg's root
> >>directory make it more clear for people outside the
> >>project?  It's a quite common practice in many projects.
> >
> >Yes, it would be handy to have.
> 
> I'd like to contribute this, but I don't know who the
> maintainers are.  Let's ask the new RM as soon as he
> gets on the bridge :-)

drivers/ati: Kevin E. Martin?, Hui Yu (ATI), Michel Dänzer (some
             components), Ben Herrenschmidt, Roland Bauerschmidt?
drivers/nv: NVidia (current liason: Andy Ritger)
drivers/sis: Thomas Winischofer
fb/: Adam Jackson (de facto)
hw/xfree86/int10: Egbert Eich
hw/xfree86/loader: Daniel Stone, Adam Jackson

et al.

> >>I'd volunteer to work on such trivial improvements
> >>for Xorg over my next holidays if there will already
> >>be a single copy of the sources to work with.
> >
> >This would be a very worthwhile tasks; marking functions with various
> >GCC attributes (noreturn, deprecated, printf, whatever) would also
> >be awesome.
> 
> It's something I could certainly do with my limited knowledge of
> the Xorg code base.
> 
> The only problem is that, work like this would have to be synced
> up between xorg/xc/ and xlibs/ therefore it's better to delay it
> until the monolithic tree finally rests in peace.

Either that, or just work on it in a branch in xlibs, so we have a
sensible delta: if we have defined deltas, merging suddenly isn't a
terribly hard problem.

That said, it's very easy to do it badly; the ANSIfication/^L work in
xlibs was very disruptive to merging, e.g.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.x.org/archives/xorg/attachments/20050216/060a072f/attachment.pgp>