[RFC PATCH 0/8] Meson build system

[Daniel Stone]

Hey Emil,

On 1 December 2016 at 14:11, Emil Velikov <emil.l.velikov at gmail.com> wrote:
> On 1 December 2016 at 11:46, Daniel Stone <daniel at fooishbar.org> wrote:
>> On Intel, the configure stage (autogen / meson -Dfoo) is 14-15x
>> faster. A complete build is 3x faster, and rebuilding component parts
>> of the tree is 1.5-4x faster. The 'rebuild weston from built tree'
>> sees one huge spike on autotools, which makes me think there's some
>> relinking going on, so the stats are potentially far worse for
>> autotools than Meson. Running tests is marginally faster, and
>> installation is 2-4x faster. On ARM, configure is ~20x faster,
>> complete build and rebuilds 3-4x faster, tests not noticeably
>> different, and install 2-4x faster (less I/O load?).
>>
>> I also updated the Wayland tree to support cross-compilation by
>> building a native scanner as part of the build process, which was
>> seriously trivial.
>>
> Thanks for the extensive tools - I merely had for i in `sed 1 3`; do
> ...; done :-)
> I'll give them a try in the next few days.
>
> As hinted before - in theory [at least] one should be able get
> noticeable improvement, by polishing the old tools - convert from
> mtime to the method used by ninja, strip down unneeded tests from
> configure, etc.

Mm, it is possible, but it still doesn't get it down to the kinds of
runtimes which I'd find actually acceptable, especially when doing
things like git rebase -i --exec 'make -C build-x86-64 check -j16' to
make sure I haven't screwed anything up at any point in my tree during
rebasing.

> This does not mitigate the fact that meson/ninja _does_ have some cool
> features and is noticeably faster _by default_.
> Yet there's also the fact that distributions and/or builders simply
> cannot use python/similar tools when using distribution tarballs.

Hm, why can't they use Python ... ? Installing Meson in a clean Debian
chroot (no Python3 installed) resulted in 5.5MB of packages being
downloaded, which expanded to 28MB on disk, much of which appears to
be Python locales, or things like expat and bzip2 which we would've
needed anyway. Fedora, Debian/Ubuntu, and OpenSUSE all have very
up-to-date versions, and I understand there's support in Yocto/BitBake
as well. It's not the sort of thing I'd expect Gentoo/Arch to have an
issue with.

For example, Debian's Weston build depends on debhelper (including
Perl), Quilt, Pixman, libpng, libjpeg, colord, all of Mesa (with a
transient build-dependency to LLVM on x86), XCB and a few component
libraries, Xlib, udev/systemd, xkbcommon, Cairo (>4MB), Pango, (2MB),
gdk-pixbuf (?!), Xcursor, mtdev, VP8, and libinput. This pretty much
eclipses the weight of the Python stdlib subset (python3-minimal on
Debian), especially when compared to the combined 4.7MB + Perl (>
25MB) + M4 required to rebuild the autotools files, or the 2.7MB (?!)
of shell/Makefile/M4 generated by autoreconf.

> Latter feature seems to be missing from meson afaict, alongside
> cscope, ctags and a few others that some people still use (I don't
> personally).

Honestly, I'd be happy to burn distribution tarballs and just use 'git
evtag' or similar; https://github.com/cgwalters/git-evtag has great
discussion on the problems that reliance on tarballs causes for people
trying to do reproducible builds. But that's a discussion for another
day ...

I didn't even realise Automake had native cscope support; for me it's
not particularly useful since I do srcdir != builddir builds, and
placing my editor root in the builddir seems very odd, but ...

> Thus do check with distro maintainers/others who ship wayland/weston,
> _if_ in the future one decides to drop the autohell setup ;-)

I'd say anyone doing cross-compilation would be very happy, since with
the attached script, all I needed to do to make cross-compiling work,
including generating a build-native wayland-scanner binary, was to add
'native: true' to a couple of places and use the below
cross-compilation definition file; with Autotools, you end up doing
two separate builds, or insane gymnastics.

> Please consider using a low traffic medium (hint not wayland-devel)
> since the question _will_ get lost.

Yeah, I'd be fine with pinging them all individually. Like I say, a
long-ish transition period to get everyone adjusted and make sure we
don't have any issues with downstreams or individual users, is
something we absolutely need. I don't want to get rid of Autotools
overnight for everyone, but I do want to only ever have to use it
again myself when I change the build, to make sure I haven't broken
it. :)

Cheers,
Daniel

[0]: Using a Debian chroot generated by qemu-bootstrap, symlinks -r,
and installing the build deps, plus a small wrapper around pkg-config
to set PKG_CONFIG_LIBDIR: cat >meson-cross-stretch-armhf.txt <<EOF &&
meson --cross-file=meson-cross-arm.txt
[binaries]
c = '/usr/bin/arm-linux-gnueabihf-gcc-6'
cpp = '/usr/bin/arm-linux-gnueabihf-g++-6'
ar = '/home/daniels/src/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.04-20130415_linux/bin/arm-linux-gnueabihf-ar'
strip = '/home/daniels/src/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.04-20130415_linux/bin/arm-linux-gnueabihf-strip'
pkgconfig = '/home/daniels/bin/arm-linux-gnueabihf-pkg-config'
#exe_wrapper = 'qemu-arm-static'

[properties]
has_function_printf = true
root = '/build/sysroots/debian-stretch-armhf'
c_args = ['--sysroot=/build/sysroots/debian-stretch-armhf']
c_link_args = ['--sysroot=/build/sysroots/debian-stretch-armhf',
'-L/build/sysroots/debian-stretch-armhf/lib',
'-L/build/sysroots/debian-stretch-armhf/lib/arm-linux-gnueabihf',
'-L/build/sysroots/debian-stretch-armhf/usr/lib',
'-L/build/sysroots/debian-stretch-armhf/usr/lib/arm-linux-gnueabihf']
cpp_args = ['--sysroot=/build/sysroots/debian-stretch-armhf']
cpp_link_args = ['--sysroot=/build/sysroots/debian-stretch-armhf',
'-L/build/sysroots/debian-stretch-armhf/lib',
'-L/build/sysroots/debian-stretch-armhf/lib/arm-linux-gnueabihf',
'-L/build/sysroots/debian-stretch-armhf/usr/lib',
'-L/build/sysroots/debian-stretch-armhf/usr/lib/arm-linux-gnueabihf']

[host_machine]
system = 'linux'
cpu_family = 'arm'
cpu = 'armv7hl'
endian = 'little'
EOF