Depending on external libraries

[Daniel Stone]

--h13GW2gLSV2TxsNR
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Thu, Mar 11, 2004 at 01:10:50PM +0100, Egbert Eich wrote:
> Kendall Bennett writes:
>  > I am not talking about API compatibility, I am talking about busted co=
de.=20
>  > Since X supports a *lot* of different platforms, it is important that=
=20
>  > those libraries included work correctly on all the supported platform,=
=20
>  > not just the ones that get the most attention in the upstream projects.
>  >=20
>  > There are few few project IMHO that are stable enough to consider just=
=20
>  > grabbing the latest source code and integrating it without serious=20
>  > testing on all the platforms that things need to work on.
>=20
> Not too much efford went into maintaining things in the=20
> extras/ directory. If one did there was always the risk
> to loose fixes when pulling in new versions from upstream.
>=20
> Only 'emergency fixes' could reasonalby be justified - if
> there was no way to obtain a fixed upstream package.
> In this case one needs to make darn sure that the problem
> is communicated to the upstream maintainers for fixing.
>=20
> We both know how much trouble we had with x86emu as it=20
> was maintained both in your repository and the XFree86 tree.

So, there's little gain if you're only doing 'emergency fixes'. I'm
tipping that doesn't occur too often, and it's definitely outweighed by
the availability of fixes from upstream in a far more rapid manner, IMO.

>  > > It just causes confusion, and it's a PITA from a distributor point
>  > > of view (bugs in Xrender that were fixed 4 releases ago, but still
>  > > aren't in XFree86, for instance).=20
>  >=20
>  > Sure, but like I said, the whole point of this new project is to=20
>  > eliminate the problem of Xrender bugs not being fixed in the X=20
>  > distribution source code. Now that lots more people will have commit=
=20
>  > access, Xrender should be fixed *inside* the X distribution itself=20
>  > relatively quickly, rather than going stale as has been the problem in=
=20
>  > the past.
>=20
> Right. This however was a release cycle issue. The fixes for Xrender
> went into the XFree86 code as soon as a committer got around to it
> - which was not always in time - but they didn't get released in
> time. The theory is now that the affected lib of Xrender can be
> released independently without having to release verything else.

That was only just one example, however. I'm sure it can, will and has
happened in other places.

> This argument however gets partly invalidated when versions of
> libs get released that depend on prerelease version of other libs.

=2E.. so don't do that.

> This however could get fixed with a little more dicipline and clearly
> marking which packages belong to the latest release.

Yes.

>  > > Right, but it defeats half the point, IMO. I will personally not
>  > > be at all happy with any non-monolithic tree that includes external
>  > > libraries. From a distributor point of view, it's caused me a hell
>  > > of a lot of pain. If you depend on external libs, why not just tell
>  > > people where to go get them?=20
>  >=20
>  > For a distributor who is going to put in the effort to apply patches a=
nd=20
>  > maintain their own 'working' versions of a the necessary libs, this is=
=20
>  > not a big deal. But for some poor schmo who loves X and just wants to =
get=20
>  > his feet wet building the latest code from X.org or whatever, it can b=
e a=20
>  > really serious problem if the code won't build or even work if that=20
>  > developer has outdated libraries.
>=20
> Right. Therefore it is necessary to clearly mark these 'external'=20
> dependencies and maintain some compatibility to earlier versions of=20
> these external APIs.
> Clearly identify those 'internal' dependencies that belong together.

I don't think necessity has been demonstrated at all. How strong a case
can you make for these internal dependencies, anyway? zlib? FreeType?
libxml? If you do much of your own compiling at all, you should probably
have these kicking around, and they really aren't difficult to intsall;
if you don't have a packaging system, a build system like jhbuild can
step in and save a lot of work.

>  > One of the things that I find important is that if you have a library=
=20
>  > that some internal piece of your code relies on, you are are much bett=
er=20
>  > using a specific version of that libraries that works for what you nee=
d=20
>  > rather than relying on some myriad of potentially incompatible externa=
l=20
>  > libraries. Unless you are building an entire distro of course - in tha=
t=20
>  > case you have your own MONOLITHIC system to solve the problem of=20
>  > integration issues - your Linux distro.
>=20
> If this is really a problem one should consider having a chat with the
> folks who are maintaining this external dependency. If this proves not
> be be feasable it may be time to look for an alternative.
> One may have to use a newer version of an external lib for two reasons:
> 1. There was a bug in an old version.
>     This is an 'oh sh***' issue that can always happen. In this case
>     an update of the version of the lib installed on the system
>     may be necessary anyway, if the bug affects the user at all.
> 2. We rely on a feature that appeared in a later version of the API.
>     In this case it is upon us to provide backward compatibility
>     (with loss of functionality) whereever this is feasable.

There are bugs in old versions. There are bugs in new versions. Bugs are
everywhere. However, IME it's generally better to have the new versions,
because then upstream gets more wide testing of his new code, and code
quality improves as you learn/discover how best to fix a certain class
of bug. Code rot isn't cool, IMO.

>  > > Duplication of libraries seems a lot like duplication of effort to
>  > > me. ;)=20
>  >=20
>  > It is more than that. My philosophy on this is that the libraries=20
>  > required by something like the X server to function correctly should b=
e=20
>  > included with that project.=20
>  >=20
>  > If FreeType is something that is required to work correctly for font=
=20
>  > rendering to work in the X server, a version of FreeType should be a p=
art=20
>  > of the X server proper. I could care less whether there is already a=
=20
>  > version of FreeType installed on the end users system as a shared=20
>  > library, or what version it is (or even that it exists and builds=20
>  > natively for the target platform). The only thing that matters is that=
=20
>  > the X server has it's own local copy that is correct for what *it* nee=
ds.=20
>  > It might be static linked into the server, or dynamically linked someh=
ow=20
>  > with a different module name and path (maybe prefix important internal=
=20
>  > libraries with something so you know they are specific to the X server=
).
>=20
> You could apply the same argument to libc or the kernel.
> The reason why the libs in extra exist is to offer some convenience
> to those who don't have these libs on their system by default.
> It is only a side effect that it helps you maneuver around stupid
> API incompatibilities and support issues that have their origin on
> SW maintained elsewhere.
> Of course problems that come from external dependencies are harder
> to track down and with more versions of this external dependency around
> are more likely to happen.
> This of course is an issue that needs to be dealt with. A big problem
> I see here is that most users will be unable to provide conclusive
> information on which versions of which lib is installed on their system.

As I said to Kendall, I believe the extras to be a problem in and of
itself. If you need API stability, talk to upstream. Most of them would
be glad to provide compatibility, especially if it's needed for
something as important as X.

I don't think external-dep problems are harder to track down: on the
contrary, the newer versions are always more widely deployed, and
they're a known quantity to upstream, not some random fork they haven't
looked at the base of since 1997. Upstreams are also usually more
willing to provide support and/or fixes for problems that occur in newer
code.

> For the libs that we control I propose to do what glibc does:
> When 'executing' the shared lib like a binary it prints out the
> version number.

Sure, we can do that. It's not terribly hard.

--=20
Daniel Stone                                            <daniel@freedesktop=
.org>
freedesktop.org: powering your desktop                http://www.freedeskto=
p.org

--h13GW2gLSV2TxsNR
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQFAUHHbcPClnTztfv0RAnqMAJ9+8bZv2J6HzwsXWxOq6xuhe12QsACfWs9i
BlY/BeOaRwkwboO/cUJZuPU=
=YQak
-----END PGP SIGNATURE-----

--h13GW2gLSV2TxsNR--