Depending on external libraries

[Egbert Eich]

Daniel Stone writes:
 > > We both know how much trouble we had with x86emu as it 
 > > was maintained both in your repository and the XFree86 tree.
 > 
 > So, there's little gain if you're only doing 'emergency fixes'. I'm
 > tipping that doesn't occur too often, and it's definitely outweighed by
 > the availability of fixes from upstream in a far more rapid manner, IMO.
 > 

This used to be different long time ago and may still differ from
package to package.

 > > Right. This however was a release cycle issue. The fixes for Xrender
 > > went into the XFree86 code as soon as a committer got around to it
 > > - which was not always in time - but they didn't get released in
 > > time. The theory is now that the affected lib of Xrender can be
 > > released independently without having to release verything else.
 > 
 > That was only just one example, however. I'm sure it can, will and has
 > happened in other places.

I'm not denying this. However I'd like to suggest to define a canonical
set of X pieces that we are hosting on fd.o.

 > 
 > > This argument however gets partly invalidated when versions of
 > > libs get released that depend on prerelease version of other libs.
 > 
 > ... so don't do that.

Please don't tell me, tell those who do.

 > 
 > > This however could get fixed with a little more dicipline and clearly
 > > marking which packages belong to the latest release.
 > 
 > Yes.
 > 
 > > 
 > > Right. Therefore it is necessary to clearly mark these 'external' 
 > > dependencies and maintain some compatibility to earlier versions of 
 > > these external APIs.
 > > Clearly identify those 'internal' dependencies that belong together.
 > 
 > I don't think necessity has been demonstrated at all. How strong a case
 > can you make for these internal dependencies, anyway? zlib? FreeType?
 > libxml? If you do much of your own compiling at all, you should probably
 > have these kicking around, and they really aren't difficult to intsall;
 > if you don't have a packaging system, a build system like jhbuild can
 > step in and save a lot of work.
 > 

I thought I made it clear that zlib, freetype etc. belong to the external
dependencies in my definition. 
Why do you keep mentioning libxml? It's gone. Some pieces have been left 
in XFree86 4.4.0 - probably by accident.

Just to state it again: most libs in extra/ are indeed external dependencies.
They are maintained outside of this project and used in different contexts.

Most other libs only make sense in the context of X. Therefore I'd call
them 'internal' dependencies. Most of them are maintained here.

 > > be be feasable it may be time to look for an alternative.
 > > One may have to use a newer version of an external lib for two reasons:
 > > 1. There was a bug in an old version.
 > >     This is an 'oh sh***' issue that can always happen. In this case
 > >     an update of the version of the lib installed on the system
 > >     may be necessary anyway, if the bug affects the user at all.
 > > 2. We rely on a feature that appeared in a later version of the API.
 > >     In this case it is upon us to provide backward compatibility
 > >     (with loss of functionality) whereever this is feasable.
 > 
 > There are bugs in old versions. There are bugs in new versions. Bugs are
 > everywhere. However, IME it's generally better to have the new versions,
 > because then upstream gets more wide testing of his new code, and code
 > quality improves as you learn/discover how best to fix a certain class
 > of bug. Code rot isn't cool, IMO.

I don't think I can fully follow you here. The new code gets as much
testing as the old code did when it was new. Since it did get widespread
testing the bugs should have been fixed - at least if there have been
maintenance releases. The new version may have these bugs fixed, too,
but it also has new features which may contain new bugs.
That's why one likes to give code good public exposure and 'soak time'
to find problems. When you are a commercial entity and you have
business customers it may even be necessary to backport bug fixes to
older versions as migrating to newer versions with new bugs is not 
feasable.

 > > 
 > > You could apply the same argument to libc or the kernel.
 > > The reason why the libs in extra exist is to offer some convenience
 > > to those who don't have these libs on their system by default.
 > > It is only a side effect that it helps you maneuver around stupid
 > > API incompatibilities and support issues that have their origin on
 > > SW maintained elsewhere.
 > > Of course problems that come from external dependencies are harder
 > > to track down and with more versions of this external dependency around
 > > are more likely to happen.
 > > This of course is an issue that needs to be dealt with. A big problem
 > > I see here is that most users will be unable to provide conclusive
 > > information on which versions of which lib is installed on their system.
 > 
 > As I said to Kendall, I believe the extras to be a problem in and of
 > itself. If you need API stability, talk to upstream. Most of them would
 > be glad to provide compatibility, especially if it's needed for
 > something as important as X.

You may be right- in theory. I have seen examples of the opposite in 
the past.

 > 
 > I don't think external-dep problems are harder to track down: on the
 > contrary, the newer versions are always more widely deployed, and

I doubt this.

 > they're a known quantity to upstream, not some random fork they haven't
 > looked at the base of since 1997. Upstreams are also usually more
 > willing to provide support and/or fixes for problems that occur in newer
 > code.

Code that has been there since 1997 and used since then may not be bug free
but one can be pretty certain that the bugs that have not been found will
not bother anybody. This may not be entirely true for security issues,
though.

 > 
 > > For the libs that we control I propose to do what glibc does:
 > > When 'executing' the shared lib like a binary it prints out the
 > > version number.
 > 
 > Sure, we can do that. It's not terribly hard.
 > 

OK.

Egbert.