[Mesa-dev] [RFC] Convert mesa to automake/libtool

[Brian Paul]

Dan Nicholson wrote:
> Brian,
> 
> I'm putting forward this request completely understanding your
> position why you don't want automake and libtool in your project.
> However, I think that mesa has outgrown the static Makefiles approach
> for a number of reasons. For a project that's grown to the complexity
> of mesa, I believe you need something that is more flexible and robust
> than the current system can provide. Eric (and I think Corbin, too)
> has a branch adding automake and libtool to the mesa repo.
> 
>   http://cgit.freedesktop.org/~anholt/mesa/log/?h=automake
> 
> I haven't looked at it in detail, but I know Eric knows what he's
> doing as he's maintained many of the autotooled xorg modules. Here are
> some of the pros and cons I see to making this change.
> 
> Pros:
> * AM_CONDITIONAL provides a clean way to separate optional parts of
> the build. The way that optional components are handled right now is
> pretty fragile and basically amounts to having lists of subdirectories
> being correct in the config file.
> 
> * For all mklib's simplicity, it is inconsistent between platforms,
> doesn't handle errors and provides only a scant amount of the features
> that libtool does. Libtool provides a robust and well tested means to
> generate libraries that handles nearly all the gory details about
> generating working binaries on many platforms. I don't think anyone
> working on mklib can claim to have the same type of knowledge about
> platforms and toolchains as the libtool developers.
> 
> * Consistency in build commands for free from automake. Right now we
> have the compiling and linking commands repeated throughout the tree
> and they're typically out of sync. I've tried to keep them in sync
> before and it was a lot of effort. With automake all you really need
> to do is tell it the CFLAGS and LIBS to use and it'll take care of the
> rest.
> 
> * Parallel make jobs just work. I've fixed so many of these race
> condition bugs, but they'd all just go away using automake. It has all
> the goop built in that people usually never think about.
> 
> * Well defined distribution for tarballs. The top-level Makefile does
> the job, but automake can make this a lot easier and more robust. It
> would also be simple to handle the generated files while also
> requiring they be included in the tarball.
> 
> * Fast source dependencies without external tools. The makedepend
> route works, but automake handles this in a faster, more robust and
> safer manner. We get a lot of people posting to the mailing list about
> build errors were the solution is "make realclean". This would solve a
> lot of those issues.
> 
> * Mindshare from xorg autotools. Many of the people here have and do
> work with the autotools via xorg, so it's not like this is a
> completely foreign beast.
> 
> * Moves the burden of build tool knowledge onto someone else. I don't
> think anyone here wants to become an expert on compilers, linkers and
> ABI for multiple platforms. It become a lot easier when you toss
> things off to libtool where people are actually spending their time
> caring about them.
> 
> * Extensive documentation available, unlike the current system which
> is pretty much ad hoc.
> 
> http://www.gnu.org/software/autoconf/manual/autoconf.html
> http://www.gnu.org/software/automake/manual/automake.html
> http://www.gnu.org/software/libtool/manual/libtool.html
> http://sourceware.org/autobook/autobook/autobook.html
> 
> Cons:
> * The abstracted nature of automake causes build debugging to be
> difficult. This requires you to train your brain not to look at the
> generated Makefile, but still it can be difficult. Fortunately, many
> of the build bugs we see today in Mesa would go away with automake.
> 
> * Using libtool means that you can't quickly hack around a fix for
> some platform. Fortunately, libtool is a lot more stable these days on
> common platforms.
> 
> * The maintainer (you) doesn't like it. Not much I can suggest here
> besides that it gets a lot easier the more you deal with it. And I'd
> be happy to help you out of any jams. For xorg, Peter Hutterer has
> asked me to solve a bunch of these problems, and I can't remember the
> last time we couldn't get something fixed.
> 
> * Loss of the simple "make $target" usage. I understand you guys use
> these targets to quickly pop out a build. As a compromise, we could
> turn the configs into wrapper scripts that generated the autotools
> files and ran configure with appropriate arguments. For example,
> "./configs/linux-debug && make". Or, since configure handles the
> platform parts, ./configs/debug or ./configs/osmesa.
> 
> That's all I can think of. I'm sure we can continue to make the
> current system work, but I think integrating these tools would be a
> big improvement. Thanks for considering it.


Hi Dan,

Thanks for the thoughtfully worded pros & cons.  Your arguments are 
compelling and I can see value in a switch to automake.  Just let me 
list my concerns.

Some years ago someone converted Mesa to use autoconf/libtool and it 
was pretty awful.

The libtool script wrapped every cc command and it slowed the build 
process considerably.  Have you done any "before/after" build time 
comparisons?  I build a *lot*.

Despite the goal of cross-platform support for autoconf/libtool, it 
actually seldom worked properly on anything but GNU/Linux.  People who 
tried to build Mesa on AIX, Ultrix, SunOS, HP-UX (including myself) 
were screwed and I wasted lots of time trying to figure out the 
problems.  In the end I usually threw my hands up in frustration and 
gave up. Meanwhile, the static configs were easy to fix and worked 
great for many years.

Probably 99% of Mesa users on Unix are using Linux or *BSD but I still 
feel some loyalty to that other 1% and would feel bad if Mesa no 
longer worked for them.

Nearly every day I get email from people with questions about Mesa 
(how to build it, use it, etc.) - stuff you never see on the mailing 
lists.  If we go the autoconf/libtool route I'd have to forward all 
such questions about building Mesa to you or someone else.  I'd be 
counting on you/someone to address issues in a timely manner.  I like 
to try to help people with their problems and I kind of hate to pass 
the buck but I wouldn't have any choice.

There are some lesser-used drivers like OSMesa and the glslcompiler 
binary that I would want supported.  Can you take care of that too? 
I'm afraid that the nooks and corners of Mesa (like those two) which 
aren't of interest to you or Eric will get ignored.

When the person who contributed autoconf support years ago disappeared 
there wasn't anyone left to take his place so I quickly and happily 
ripped out all that stuff and set up the static build system.  I doubt 
that'd happen again, but that's what happend last time.

I dread being in situations where something's broken and I can't fix 
it myself.  That's one of my main issues with the proposed change.


So anyway, I just checked out Eric's git tree and the automake branch.
I ran "./autogen.sh":

$ ./autogen.sh
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal -I m4
aclocal: couldn't open directory `m4': No such file or directory
autoreconf: aclocal failed with exit status: 1

Off to a flying start, huh?  At this point I have no clue what's 
wrong.  If this were Mesa/master I'd be totally stuck and feeling 
helpless (as I usually do when I try to build X.org).  Maybe Eric's 
branch isn't in a buildable state.  Is there another tree/branch that 
I should try out?

-Brian