<div dir="ltr">Hi Daniel,<div> </div><div>Thanks for doing this!</div><div> </div><div>I share a lot of the same background - I've worked with autotools for many years, know them well and have also been hesitant to move away. I've recently ported a small personal project from autotools to meson and I found it very nice and easy to work with. It feels like a better fit for the kind of workflow that I'm used to with autotools than many of the other newer build systems out there.</div><div> </div><div>In terms of speed, it's certainly nice that meson ends up using ninja, but the much faster configure step is what I'm excited about. Especially on a multicore system where the configure step runs single-threaded and bottlenecks the rest of the otherwise parallelizable build process. And the fact that meson is fast enough that there's no point in a separate autogen.sh step, make the overall system much simpler and faster.</div><div> </div><div>Of course, one of the points of the generate step in autotools is that you ship the generated files with your tarballs and the end results only requires standard sh and make. I don't think meson has a similar feature, but I'd argue that today requiring python isn't going to limit you any more than requiring sh and make.</div><div> </div><div>I think having both meson and autotools side by side for a bit would be a good way to start and I'd be in favor of moving to meson exclusively a few releases down the road. </div><div> </div><div>Kristian</div><div> <div class="gmail_quote"><div dir="ltr">On Tue, Nov 29, 2016 at 9:00 AM Daniel Stone <<a href="mailto:daniels@collabora.com">daniels@collabora.com</a>> wrote: </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi all, Whilst working on the atomic series, and the recent review spree, I started to really lose patience with how incredibly slow our build system is. This patchset provides a working port to Meson, a Python-based build system with a Ninja backend. Whilst autotools runs a combination of shell and Perl (aclocal/automake/autoconf) to generate the build files and Make and shell to run it, Meson runs its configuration step in Python, and outputs Ninja build files. (tl;dr: smaller platforms 3-6x quicker, bigger platforms 2-4x quicker) There are a few things which attracted me to Meson. It's really, mindblowingly, fast. There's no artificial split between configure and build stages (i.e. <a href="http://configure.ac" rel="noreferrer" class="gmail_msg" target="_blank">configure.ac</a> vs. Makefile.am). Its dependencies are quite lightweight. It has first-class support for pkg-config, and I've not had a problem with executables, shared libraries, modules, static helper libraries, binaries to install to libexec, tests[0], etc etc. I've been keeping an eye on it for a while as it matures; one of the things which interested me was seeing GStreamer and GNOME start to migrate some of their projects towards it. Not because they're incredible and we should blindly follow everything they do, but because they have a very similar set of build-system requirements to us, and they have also been very conservative in migrating away from autotools to date. I'm incredibly familiar with autotools, dating back to 2004 when I started experimenting with converting the X server to use it, which later grew into the xserver module we have today. As new build systems have come around, I've made a point of looking into them, and trying to use them. SCons and waf were immediately disqualified, because I think writing your entire build in actual Python is insane, and too large a barrier to entry. CMake is brutally slow[1], I find its interface very unintuitive, and cmake --help-full splits out a 37,399-line Markup file to stdout. GN (used for Chrome) is very nice, and I prefer its system of 'configs' to Meson's per-target flags, but isn't really portable. Most of the rest focus on languages we don't use, and/or bring massive dependencies. Meson's the first one I've found that strikes the right balance (not perfect, but within striking distance) of lightweight DSL vs. useful programming language. And, it's really incredibly fast. It has far better support for cross-compilation and sysroots than autotools can ever hope to have. And it's got quite a good upstream community behind it, and solid documentation. I've done a port for Meson, with the full range of options and configuration available in autotools today. Currently we're missing Doxygen support (because I haven't got around to it) in Wayland, plus I haven't been able to test the Weston RDP backend (because it doesn't build in Fedora 25) or the fbdev backend (because it's 2016), but they are there. There's also currently no support for 'auto' build options; I need to look into how best to do that. A couple of the tests in Weston are missing, and I think also PAM support for weston-launch. There are also some inefficiencies: currently, for some reason, Meson will run wayland-scanner once for every generated file _for every target_. So xdg-shell-unstable-v5-client-protocol.h, for example, gets built a bazillion times. Fixing that upstream would actually visibly cut the build times. We could also get a speed boost by moving away from config.h and using per-target build flags, to avoid unnecessary rebuilds: this doesn't work with autotools as it only rebuilds if dependent mtimes change, but Meson/Ninja hash the build flags and force a rebuild if those change. I'm going to be looking into that later. It's also smaller than our existing files: 374 lines for Meson versus 481 lines for <a href="http://configure.ac/Makefile.am" rel="noreferrer" class="gmail_msg" target="_blank">configure.ac/Makefile.am</a> (excluding doc) in Wayland, and 1602 lines for Meson versus 2373 lines for <a href="http://configure.ac/Makefile.am/weston-tests-env" rel="noreferrer" class="gmail_msg" target="_blank">configure.ac/Makefile.am/weston-tests-env</a> in Weston. This conversion requires Meson 0.36, which is available in Fedora 25 and Debian testing/stretch. It's not available in Ubuntu release versions or Debian stable/jessie; it can, however, be easily installed through pip on these systems. Thanks for this go to the very helpful people on Freenode #mesonbuild, as well as to the authors of the GStreamer Meson port, who stated that they did not assert copyrightability over their build files, allowing me to use some of those as inspiration for this port. Anyway, here are some benchmarks I've done on build times, across the full range of the computing-power spectrum: System 1 ('RPi2'): Raspberry Pi 2, 900MHz quad-core ARM Cortex-A7, 1GB RAM, storage on SD card, make -j8 System 2 ('Rock2'): Radxa Rock2 Square, Rockchip RK3288 SOC, 1.6GHz quad-core ARM Cortex-A17, 2GB RAM, storage on eMMC, make -j8 System 3 ('Laptop'): Dell XPS 13 9350, 2.5GHz dual-core Intel Skylake i7-6500U, 4MB cache, 16GB RAM, storage on NVME/PCI-E, make -j8 System 4 ('Xeon'): HP Z840, 2.4GHz 14-core Intel Broadwell Xeon E5-2680v4, 35MB cache, 128GB RAM, storage on NVME/PCI-E, make -j18[2] Wayland ------- Documentation is disabled to give an apples-to-apples comparison; also Meson's default use of ccache is disabled for the same reason. 'make install' is split into a separate make invocation, because it fails in a dependency-relinking catastrophe otherwise. Note that the time involving test is mostly dominated by sanity-test, which takes around 5.7s (!). If you subtract that from the runtimes, the differentials for the faster machines becomes far more apparent. Test 1 ('full'): full configuration, build, test, and install - time (../autogen.sh --prefix=/tmp/wayland-autotools --disable-documentation && make -j$JOBS check && make install) - time (CC=gcc meson --prefix=/tmp/wayland-meson .. && ninja test install) Test 2 ('rebuild'): full rebuild and install - touch ../src/wayland-util.h && time (make -j$JOBS && make check install) - touch ../src/wayland-util.h && time (ninja test install) Test 3 ('check'): relink libweston and test - touch ../src/wayland-server.c && time (make -j$JOBS check) - touch ../src/wayland-server.c && time (ninja test) Test 4 ('install'): no changes, install - time make install - time ninja install | RPi2 | Rock2 | Laptop | Xeon | ---------+------------------+-----------------+-----------------+----------------+ Full | 353.34s / 65.78s | 96.80s / 25.30s | 27.78s / 11.24s | 22.71s / 7.19s | Rebuild | 103.73s / 53.83s | 43.61s / 16.17s | 13.80s / 9.63s | 10.87s / 6.12s | Check | 44.67s / 14.76s | 17.63s / 7.23s | 13.22s / 8.80s | 6.67s / 5.41s | Install | 6.39s / 3.46s | 1.71s / 1.13s | 0.47s / 0.14s | 0.75s / 0.36s | ---------+------------------+-----------------+-----------------+----------------+ Weston ------- Documentation is disabled to give an apples-to-apples comparison; also Meson's default use of ccache is disabled for the same reason. 'make install' is split into a separate make invocation, because it fails in a dependency-relinking catastrophe otherwise. The RDP backend is disabled for both, so I can actually build it on Fedora. Test 1 ('full'): full configuration, build, test, and install - time (../autogen.sh --prefix=/tmp/weston-autotools --disable-rdp-compositor --disable-devdocs --disable-setuid-install && make -j$JOBS check && make install) - time (CC=gcc meson --prefix=/tmp/weston-meson -Dbackend_rdp=false .. && ninja test install) + -Dclients_dmabuf_drm=false / --disable-simple-dmabuf-intel-client on the ARM systems Test 2 ('rebuild'): full rebuild and install - touch ../libweston/compositor.h && time (make -j$JOBS && make check install) - touch ../libweston/compositor.h && time (ninja test install) Test 3 ('check'): relink libweston and test - touch ../libweston/timeline.c && time (make -j$JOBS check) - touch ../libweston/timeline.c && time (ninja test) Test 4 ('install'): no changes, install - time make install - time ninja install | RPi2 | Rock2 | Laptop | Xeon | ---------+-------------------+------------------+-----------------+----------------+ Full | 685.80s / 163.20s | 188.55s / 34.55s | 53.96s / 12.47s | 25.78s / 6.26s | Rebuild | 231.78s / 77.62s | 62.26s / 19.45s | 25.75s / 6.81s | 14.25s / 4.80s | Check | 37.21s / 27.18s | 12.00s / 6.82s | 4.91s / 3.22s | 4.42s / 3.65s | Install | 21.32s / 7.31s | 7.03s / 1.98s | 1.63s / 0.38s | 2.23s / 0.73s | ---------+-------------------+------------------+-----------------+----------------+ Cheers, Daniel [0]: Support for skipping tests has still not quite yet landed, which is a black mark indeed: <a href="https://github.com/mesonbuild/meson/pull/1110" rel="noreferrer" class="gmail_msg" target="_blank">https://github.com/mesonbuild/meson/pull/1110</a> [1]: cf. <a href="https://lwn.net/Articles/706653/" rel="noreferrer" class="gmail_msg" target="_blank">https://lwn.net/Articles/706653/</a> and thread [2]: This would be -j28, but the build just fails at higher parallelism levels. I don't know why. _______________________________________________ wayland-devel mailing list <a href="mailto:wayland-devel@lists.freedesktop.org" class="gmail_msg" target="_blank">wayland-devel@lists.freedesktop.org</a> <a href="https://lists.freedesktop.org/mailman/listinfo/wayland-devel" rel="noreferrer" class="gmail_msg" target="_blank">https://lists.freedesktop.org/mailman/listinfo/wayland-devel</a> </blockquote></div></div></div>