Depending on external libraries

[Egbert Eich]

Kendall Bennett writes:
 > I am not talking about API compatibility, I am talking about busted code. 
 > Since X supports a *lot* of different platforms, it is important that 
 > those libraries included work correctly on all the supported platform, 
 > not just the ones that get the most attention in the upstream projects.
 > 
 > There are few few project IMHO that are stable enough to consider just 
 > grabbing the latest source code and integrating it without serious 
 > testing on all the platforms that things need to work on.

Not too much efford went into maintaining things in the 
extras/ directory. If one did there was always the risk
to loose fixes when pulling in new versions from upstream.

Only 'emergency fixes' could reasonalby be justified - if
there was no way to obtain a fixed upstream package.
In this case one needs to make darn sure that the problem
is communicated to the upstream maintainers for fixing.

We both know how much trouble we had with x86emu as it 
was maintained both in your repository and the XFree86 tree.

 > 
 > > It just causes confusion, and it's a PITA from a distributor point
 > > of view (bugs in Xrender that were fixed 4 releases ago, but still
 > > aren't in XFree86, for instance). 
 > 
 > Sure, but like I said, the whole point of this new project is to 
 > eliminate the problem of Xrender bugs not being fixed in the X 
 > distribution source code. Now that lots more people will have commit 
 > access, Xrender should be fixed *inside* the X distribution itself 
 > relatively quickly, rather than going stale as has been the problem in 
 > the past.

Right. This however was a release cycle issue. The fixes for Xrender
went into the XFree86 code as soon as a committer got around to it
- which was not always in time - but they didn't get released in
time. The theory is now that the affected lib of Xrender can be
released independently without having to release verything else.

This argument however gets partly invalidated when versions of
libs get released that depend on prerelease version of other libs.

This however could get fixed with a little more dicipline and clearly
marking which packages belong to the latest release.


 > > Right, but it defeats half the point, IMO. I will personally not
 > > be at all happy with any non-monolithic tree that includes external
 > > libraries. From a distributor point of view, it's caused me a hell
 > > of a lot of pain. If you depend on external libs, why not just tell
 > > people where to go get them? 
 > 
 > For a distributor who is going to put in the effort to apply patches and 
 > maintain their own 'working' versions of a the necessary libs, this is 
 > not a big deal. But for some poor schmo who loves X and just wants to get 
 > his feet wet building the latest code from X.org or whatever, it can be a 
 > really serious problem if the code won't build or even work if that 
 > developer has outdated libraries.

Right. Therefore it is necessary to clearly mark these 'external' 
dependencies and maintain some compatibility to earlier versions of 
these external APIs.
Clearly identify those 'internal' dependencies that belong together.

 > 
 > One of the things that I find important is that if you have a library 
 > that some internal piece of your code relies on, you are are much better 
 > using a specific version of that libraries that works for what you need 
 > rather than relying on some myriad of potentially incompatible external 
 > libraries. Unless you are building an entire distro of course - in that 
 > case you have your own MONOLITHIC system to solve the problem of 
 > integration issues - your Linux distro.

If this is really a problem one should consider having a chat with the
folks who are maintaining this external dependency. If this proves not
be be feasable it may be time to look for an alternative.
One may have to use a newer version of an external lib for two reasons:
1. There was a bug in an old version.
    This is an 'oh sh***' issue that can always happen. In this case
    an update of the version of the lib installed on the system
    may be necessary anyway, if the bug affects the user at all.
2. We rely on a feature that appeared in a later version of the API.
    In this case it is upon us to provide backward compatibility
    (with loss of functionality) whereever this is feasable.

 > 
 > > Duplication of libraries seems a lot like duplication of effort to
 > > me. ;) 
 > 
 > It is more than that. My philosophy on this is that the libraries 
 > required by something like the X server to function correctly should be 
 > included with that project. 
 > 
 > If FreeType is something that is required to work correctly for font 
 > rendering to work in the X server, a version of FreeType should be a part 
 > of the X server proper. I could care less whether there is already a 
 > version of FreeType installed on the end users system as a shared 
 > library, or what version it is (or even that it exists and builds 
 > natively for the target platform). The only thing that matters is that 
 > the X server has it's own local copy that is correct for what *it* needs. 
 > It might be static linked into the server, or dynamically linked somehow 
 > with a different module name and path (maybe prefix important internal 
 > libraries with something so you know they are specific to the X server).

You could apply the same argument to libc or the kernel.
The reason why the libs in extra exist is to offer some convenience
to those who don't have these libs on their system by default.
It is only a side effect that it helps you maneuver around stupid
API incompatibilities and support issues that have their origin on
SW maintained elsewhere.
Of course problems that come from external dependencies are harder
to track down and with more versions of this external dependency around
are more likely to happen.
This of course is an issue that needs to be dealt with. A big problem
I see here is that most users will be unable to provide conclusive
information on which versions of which lib is installed on their system.

For the libs that we control I propose to do what glibc does:
When 'executing' the shared lib like a binary it prints out the
version number.

Cheers,

	Egbert.