[RFC PATCH 0/8] Meson build system

[Daniel Stone]

Hey Emil,
Thanks for the detailed reply! :) It's really interesting to hear your
perspective, especially with Mesa also using SCons.

On 30 November 2016 at 01:02, Emil Velikov <emil.l.velikov at gmail.com> wrote:
> On 29 November 2016 at 20:50, Daniel Stone <daniel at fooishbar.org> wrote:
>>> As you know better than me the actual speed increase isn't in using
>>> Meson, it's due to ninja.
>>>
>>> If one is to use (write?) make backend for Meson the results wouldn't
>>> be that different.
>>
>> I disagree. Using Ninja is nice, and a huge win, but there is already
>> a massive, massive, massive, win before you invoke either Make or
>> Ninja; just generating the configuration (autoreconf + ./configure vs.
>> meson) is 16.75 seconds on my laptop (again, current-generation Intel
>> with 16GB of RAM) for autotools, vs. 1.24 seconds for Meson. If
>> they're equal, we'd have 53.96s vs. 28.0sec for the total build,
>> rather than 53.96 vs. 12.47.
>>
> Thanks for the reminder how much configure sucks and how much faster
> non-recursive makefiles are.

Heh, Wayland and Weston both (apart from docs) use non-recursive
Makefiles! We took the readability hit of flattening everything into
one Makefile so we could get a speed/parallelism boost.

> Numbers seems far better over here...
>
> $ git clean -fXd -- .. && rm -rf * && time (../autogen.sh
> --prefix=/tmp/wayland-autotools --disable-documentation)
> ~10 real
>
> The rest of the number also seems to vary noticeably, despite having a
> lower spec machine.
> Using stock Arch here, should distro of choice matter that much ?
>
> System 3 ('Laptop'): Dell XPS 13 9350, 2.5GHz dual-core Intel Skylake
> i7-6500U, 4MB cache, 16GB RAM, storage on NVME/PCI-E, make -j8
>
> System X ('Laptop 2'): Lenovo X1C3, 245GHz dual-core Intel Broadwell
> i7-5500U, 4MB cache, 8GB RAM, storage on NVME/PCI-E, make -j8
>
>
> NB: Everything but the Install test, varies ±0.2s across 3 consecutive
> runs, thus it's been rounded to the closest 0.5s on average.
>
>          |          Laptop |       Laptop 2 |
> ---------+-----------------+----------------+
> Full     | 27.78s / 11.24s |   ~21s /  ~10s |
> Rebuild  | 13.80s /  9.63s | ~12.5s / ~8.5s |
> Check    | 13.22s /  8.80s |  ~7.5s / ~5.5s |
> Install  |  0.47s /  0.14s |  0.47s / 0.14s |
> ---------+-----------------+----------------+

Ha, this isn't Skylake laptop vs. Broadwell laptop, this is my 14-core
Xeon vs. your Broadwell laptop! That they're so different suggests
that our configuration is pretty wildly different - are you perhaps
not building a load of stuff which configure autodetects? That's the
only way I can imagine how your laptop would outpace a £4000
workstation. :\

> Having a close look at the functional discrepancies:
>  - meson/ninja does not create static libraries, like autotools
> Haven't tried unifying/checking.

Indeed. To be clear, Meson itself can, but I'm just not doing that
yet. I think we could get a further boost, especially on the slower
machines, by generating those intermediate static libraries. That
would be a reasonable next step, but I wanted to get this out there
before -ETIME, given that this has been a spare-time project for me.

>  - meson checks for build_docs programs (toggle build_docs to false in
> meson_options.txt)
> Changing it makes no noticeable difference.
>  - autotools builds fixed-benchmark
> Removing it makes no noticeable difference.

Oh, yes. There are also, as mentioned, a couple of tests which aren't
yet built in Meson.

> So if we are to ignore configure time, numbers are comparable.
> And yes, that's a very big if to ask for.

Interesting indeed. I do wonder what is going on there. Either way,
it's still a 1.5-2x speed win for you ...

As another data point, when I'm rebasing, I make very heavy use of git
rebase -i --exec right down to the parent tree, to make sure it
compiles/works the whole way through. Across my compositor-drm tree
(now up to 60 patches; most only touching compositor-drm.c, but some
touching wider parts of the source), this was 260.0s for make -j8, and
19.3sec for ninja. There were three points at which autotools re-run
and Meson didn't; if we assume we have to take the full penalty for a
rebuild, then we end up on ~50s vs. ~260s.

>> Ninja works on cores*2 by default AIUI, to match either the number of
>> threads (relevant for my Intel machines). Even in the RPi case where
>> it's not matching threads, it at least allows you to hide I/O latency
>> a little bit, which is huge on an SD card ...
>>
>>> At the same time, -jJOBS seems to be missing in some of the make
>>> invocations. Is that just in the summary here in the actual test runs
>>> as well ?
>>
>> It's missing in make install, because parallelising
>>
> $ make -j100 check && make -j100 install
> ... works like a charm.

Hm, I'm sure that didn't work when I tried it on the Xeon at least,
but either way, on my laptop it drops install time from ~1.7s to
~1.4s, stable over a few runs.

> I've never tried doing both check and install targets at the same time
> and I'm not sure if its legal.

Indeed it doesn't seem to be. Build and check together seem to be
fine, but running install in parallel with anything else always ends
in a relinking catastrophe.

>> Slower again? How?!
>>
> Not sure what you mean here.
>
> As we build (simple `make') binaries are linked against the in-tree
> DSO(s). Upon `make install' autotools/libtool relinks each binary. The
> latter of which can be quite costly.
> One way around it is to hack into libtool with another to use a
> project called slibtool (I've mentioned it a while back). The latter
> links only once, thus in theory (at least) the whole thing could be
> noticeably faster. For some reason normal linking with slibtool can be
> a bit slow :-(

Having a slower runtime than libtool (i.e. 'make clean && make -jN
all' being 50% slower with slibtool) is strange; I thought the entire
reason slibtool existed was to be faster than libtool. But yes, I
would assume that relinking accounts for a large part of the
performance differential on the installation step for ARM at least.

>> Having worked with autotools quite a lot for the last 12 years, I
>> can't say I was ever tempted to send a patch for it. Even just working
>> around some of its more esoteric quirks and deficiencies (cf. magic
>> relinking rules which kill parallelism) makes me lose the will to
>> live. On the other hand, I found Meson very easy and accessible to
>> work with, and had the pull request for skipping tests accepted today.
>> I just can't ever imagine bringing myself to work on the beast which
>> is autotools.
>>
>> Plus, what are you realistically going to do with it which would
>> result in a 2-6x speed increase? It's used so widely that I cannot for
>> the life of me imagine that it's only because of no-one working on it
>> that it's this slow. I'm going to ascribe it more to being enormous
>> piles of shell and Perl, compatible all the way back to some ancient
>> AIX sh variant. Might be wrong, but ...
>>
>> Point is, autotools has literal decades to be this fast, and still
>> isn't. To me, that points to deeper issues.
>>
> Was hoping that you've seen some threads and/or have some experience to share.

Ugh, sorry, re-reading it, that comes off as really dismissive. Please
accept my apologies: it wasn't what I meant. Maybe I should stop
writing mails in a bit of a hurry so I can go eat dinner. ;)

> At the same time I your statement [above] states what I've find most
> of the time.
> People being unhappy with autotools, yet lacking the time/interest to
> improve it :'-(
>
> Please don't get me wrong, autohell^Wautotools is pretty annoying/etc
> at times. Yet again I'm more of a person who would aim to improve
> things before going to plan B.

Yeah, generally I agree. That was my point about GNOME/GStreamer also
being conservative with build systems: neither of us have jumped over
to any new and fancy build systems before, because the benefits just
didn't outweigh the disadvantages of doing so. And I think that makes
total sense, since switching does have a very large cost. For all that
autotools is a Kafkaesque nightmare, and no-one actually knows how M4
quoting works, it's well understood by end users who build, by
distributors and integrators who build, and by developers who work on
the code. That's a huge thing to be throwing away, and not to be done
lightly.

Meson for me is the first system whose advantages actually outweigh
the disadvantage of losing that. But that being said, I would like the
two to sit in parallel for a couple of releases; I don't think it
would make much sense to remove autotools support before 1.16-1.17,
when we'll have a useful version of Meson in the released version of
Debian (stretch) and Ubuntu, soon before that also becomes the new
Ubuntu LTS version. That would give us as developers plenty of time to
get used to things, and downstream plenty of time to adjust and help
us discover problems with it.

> Since wayland/weston uses non-recursive makefiles it's a matter of a)
> removing the relinking and b) beating some sense into configure
> (hiding most of the legacy tests?).
> I'm afraid there's no 2-6x speed difference here and with the above
> one could easily match (even surpass?) meson/ninja in some/most tests.
>
> As a closing though: I wonder if I would be so nice and calm as you,
> if I had used it for ~12 years.
> I'm pretty sure things weren't as easy, in the yearly days, as they are today.

I think I just had a lot more time on my hands ... ?

Thanks again for the background and consideration here!

Cheers,
Daniel