Depending on external libraries

[Daniel Stone]

--RacQGezy2Y99S6cT
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Wed, Mar 10, 2004 at 05:36:12PM -0800, Kendall Bennett wrote:
> Daniel Stone <daniel@freedesktop.org> wrote:
> > > IMHO if the X tree *requires* and depends on certain external librari=
es=20
> > > in order to work correctly, those pieces should be included in the en=
tire=20
> > > X tree and built as part of that system. Sure it is more work to impo=
rt=20
> > > the external code into the X tree on a regular basis, but the key poi=
nt=20
> > > is that the code that finally ends up in the resulting release tree h=
as=20
> > > been through the same release and testing process as the rest of the=
=20
> > > code. That way you don't end up with serious gotchas in the field wit=
h=20
> > > varying versions of external pieces being used.=20
> >=20
> > I do not believe this to be the case. It just causes heaps of
> > heartache for completely hypothetical situations when, say, there's
> > a security bug in zlib or something equally improbable. If you
> > depend on a specific version because of API reasons, talk to
> > upstream! If one part of the API must be absolutely rock-solid, ask
> > them to keep compatibility wrappers around.=20
>=20
> I am not talking about API compatibility, I am talking about busted code.=
=20
> Since X supports a *lot* of different platforms, it is important that=20
> those libraries included work correctly on all the supported platform,=20
> not just the ones that get the most attention in the upstream projects.

Yes, but X is a special case, doing the amount of OS and system
interaction it does. Most code doesn't need to care about portability,
as long as it's 64-bit clean, and the underlying libc is sane. If I
write a library to read a configuration file, for instance, it's fairly
safe to assume that strstr, strcpy, malloc and friends are all safe[0].

> There are few few project IMHO that are stable enough to consider just=20
> grabbing the latest source code and integrating it without serious=20
> testing on all the platforms that things need to work on.

Sure, I agree with you completely. I'm not suggesting we go upgrade to
CVS willy-nilly.

> > It just causes confusion, and it's a PITA from a distributor point
> > of view (bugs in Xrender that were fixed 4 releases ago, but still
> > aren't in XFree86, for instance).=20
>=20
> Sure, but like I said, the whole point of this new project is to=20
> eliminate the problem of Xrender bugs not being fixed in the X=20
> distribution source code. Now that lots more people will have commit=20
> access, Xrender should be fixed *inside* the X distribution itself=20
> relatively quickly, rather than going stale as has been the problem in=20
> the past.

Right, but at that point you're still duplicating effort. If I work on
this configuration lib, I'm committing to my original sources, plus X.
It's a pain, and keeping it in sync will be a nightmare. It's already
proven to be a nightmare - throwing more people at the problem won't
make it go away.

> > > Note also that the policy of including all the source to build the=20
> > > necessary pieces does not necessarily infringe on the idea of a modul=
ar=20
> > > build system either. You only need to keep around the pieces required=
 by=20
> > > each modular part of the system as necessary. The key thing is to mak=
e=20
> > > sure stuff is broken up properly and that stuff is properly integrati=
on=20
> > > tested together as a unit.
> >=20
> > Right, but it defeats half the point, IMO. I will personally not
> > be at all happy with any non-monolithic tree that includes external
> > libraries. From a distributor point of view, it's caused me a hell
> > of a lot of pain. If you depend on external libs, why not just tell
> > people where to go get them?=20
>=20
> For a distributor who is going to put in the effort to apply patches and=
=20
> maintain their own 'working' versions of a the necessary libs, this is=20
> not a big deal. But for some poor schmo who loves X and just wants to get=
=20
> his feet wet building the latest code from X.org or whatever, it can be a=
=20
> really serious problem if the code won't build or even work if that=20
> developer has outdated libraries.

Last I checked, it was 110,000 lines of patches for Debian. That's a
pretty big deal.

Stuff like jhbuild makes this quite easy, by fulfilling dependencies. If
they're clearly expressed, the problem largely goes away, and you don't
have to deal with the duplication of effort. Also, if you're going to be
doing any development, it should be your responsibility to maintain your
box and get it into a decent shape. We shouldn't be shipping an IDE or
something, on the basis that devs shouldn't have to install it - so why
do we fork FreeType?

> One of the things that I find important is that if you have a library=20
> that some internal piece of your code relies on, you are are much better=
=20
> using a specific version of that libraries that works for what you need=
=20
> rather than relying on some myriad of potentially incompatible external=
=20
> libraries. Unless you are building an entire distro of course - in that=
=20
> case you have your own MONOLITHIC system to solve the problem of=20
> integration issues - your Linux distro.

Sure. So, in that case, you either write upstream and ask them to pretty
please keep some compatibility around (seriously, most people take
requests from X hackers, about X, reasonably seriously), or you don't
use a library that's that fluid. Of course, the second solution is quite
utopian, so it can usually be safely discounted.

> It really depends entirely on whether you want the X server project to be=
=20
> something standalone, or something that should only ever work when=20
> delivered as part of a complete OS distribution. You guys keep arguing=20
> about the fact that you don't like a monolithic server but instead want=
=20
> modular trees and modular libraries, yet at the end of the day you are=20
> building the biggest monolithic system of all - your OS!
>=20
> Kind of ironic eh ;-)

Not really, no. The various packages in Debian (about 8,000, last I
checked), are all disconnected from each other - in quite a modular way,
you might say. libfreetype1 doesn't have to care about zlib1g, yet they
interoperate beautifully.

I think it's reasonable to assume a pretty complete system if you're
building X. It's not something the average person does, or should have
to do[1], and if you're devloping, then getting your box up to scratch
is your responsibility.

I don't think we should cause ourselves significant troubles trying to
compensate for lazy would-be developers, or OSs that can't ship a recent
enough version of drivers (like one I can think of ...).


> > Duplication of libraries seems a lot like duplication of effort to
> > me. ;)=20
>=20
> It is more than that. My philosophy on this is that the libraries=20
> required by something like the X server to function correctly should be=
=20
> included with that project.=20

To what point? libc? The kernel? ld.so? A compiler? zlib? (Hey, wait
=2E..)

> If FreeType is something that is required to work correctly for font=20
> rendering to work in the X server, a version of FreeType should be a part=
=20
> of the X server proper. I could care less whether there is already a=20
> version of FreeType installed on the end users system as a shared=20
> library, or what version it is (or even that it exists and builds=20
> natively for the target platform). The only thing that matters is that=20
> the X server has it's own local copy that is correct for what *it* needs.=
=20
> It might be static linked into the server, or dynamically linked somehow=
=20
> with a different module name and path (maybe prefix important internal=20
> libraries with something so you know they are specific to the X server).

Why should FreeType be built in? Why would you be lumbering yourself
with a fork for no good reason? Risk code rot? It makes things harder
for the FreeType developers, harder for the users, harder for the X
developers, harder for everyone except hackers who can't be bothered
running apt-get install libfreetype-dev, or installing a simple tarball.

You get things like security issues, where you find you have to fix them
in 13 non-obvious places (*cough*zlib*cough*), because all these
projects have included their own forks. You get code rot and bugs that
stay around for ages despite having been long-ago fixed by upstream
(*cough*Xrender*cough*).

You're proposing an (IMO bad) fix to a mere symptom of the problem (code
rot, et al), in saying 'well, we'll be more open, so people will be able
to maintain the fork better'. The real problem is that the fork has to
be maintained at all, and at the root of the problem lies monolithic
trees.

> At the end of the day it is a trade off really. Compatibility against=20
> saving bytes on disk and memory by only having one copy of FreeType=20
> installed on the target system. I always prefer the former.

The two are not mutually exclusive. My modular builds have been working
fine for some time. Well, they haven't, but the issues have been that
autotooling the XFree86 DDX is painful due to the amount of build system
cruft, not to do with external libs.

[0]: Platforms without these have far more serious problems.
[1]: If they have to, the OS is IMO failing them. If they want to hack
     on it, then sorting out their box is their own problem.

--=20
Daniel Stone                                            <daniel@freedesktop=
.org>
freedesktop.org: powering your desktop                http://www.freedeskto=
p.org

--RacQGezy2Y99S6cT
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQFAUG9dcPClnTztfv0RApd8AJ9FsZT6hzuJRgFz6N37pXsfde+chgCeLoSG
3KU6kB/7RC4uHUZIiL2Slaw=
=qWIs
-----END PGP SIGNATURE-----

--RacQGezy2Y99S6cT--