[RFC PATCH 0/8] Meson build system

[Emil Velikov]

On 1 December 2016 at 15:24, Daniel Stone <daniel at fooishbar.org> wrote:
> Hey Emil,
>
> On 1 December 2016 at 14:11, Emil Velikov <emil.l.velikov at gmail.com> wrote:
>> On 1 December 2016 at 11:46, Daniel Stone <daniel at fooishbar.org> wrote:
>>> On Intel, the configure stage (autogen / meson -Dfoo) is 14-15x
>>> faster. A complete build is 3x faster, and rebuilding component parts
>>> of the tree is 1.5-4x faster. The 'rebuild weston from built tree'
>>> sees one huge spike on autotools, which makes me think there's some
>>> relinking going on, so the stats are potentially far worse for
>>> autotools than Meson. Running tests is marginally faster, and
>>> installation is 2-4x faster. On ARM, configure is ~20x faster,
>>> complete build and rebuilds 3-4x faster, tests not noticeably
>>> different, and install 2-4x faster (less I/O load?).
>>>
>>> I also updated the Wayland tree to support cross-compilation by
>>> building a native scanner as part of the build process, which was
>>> seriously trivial.
>>>
>> Thanks for the extensive tools - I merely had for i in `sed 1 3`; do
>> ...; done :-)
>> I'll give them a try in the next few days.
>>
>> As hinted before - in theory [at least] one should be able get
>> noticeable improvement, by polishing the old tools - convert from
>> mtime to the method used by ninja, strip down unneeded tests from
>> configure, etc.
>
> Mm, it is possible, but it still doesn't get it down to the kinds of
> runtimes which I'd find actually acceptable, especially when doing
> things like git rebase -i --exec 'make -C build-x86-64 check -j16' to
> make sure I haven't screwed anything up at any point in my tree during
> rebasing.
>
>> This does not mitigate the fact that meson/ninja _does_ have some cool
>> features and is noticeably faster _by default_.
>> Yet there's also the fact that distributions and/or builders simply
>> cannot use python/similar tools when using distribution tarballs.
>
> Hm, why can't they use Python ... ? Installing Meson in a clean Debian
> chroot (no Python3 installed) resulted in 5.5MB of packages being
> downloaded, which expanded to 28MB on disk, much of which appears to
> be Python locales, or things like expat and bzip2 which we would've
> needed anyway. Fedora, Debian/Ubuntu, and OpenSUSE all have very
> up-to-date versions, and I understand there's support in Yocto/BitBake
> as well. It's not the sort of thing I'd expect Gentoo/Arch to have an
> issue with.
>
> For example, Debian's Weston build depends on debhelper (including
> Perl), Quilt, Pixman, libpng, libjpeg, colord, all of Mesa (with a
> transient build-dependency to LLVM on x86), XCB and a few component
> libraries, Xlib, udev/systemd, xkbcommon, Cairo (>4MB), Pango, (2MB),
> gdk-pixbuf (?!), Xcursor, mtdev, VP8, and libinput. This pretty much
> eclipses the weight of the Python stdlib subset (python3-minimal on
> Debian), especially when compared to the combined 4.7MB + Perl (>
> 25MB) + M4 required to rebuild the autotools files, or the 2.7MB (?!)
> of shell/Makefile/M4 generated by autoreconf.
>
The above seems like a strange mix of build tools and (runtime) dependencies.

Not sure that comparing X and Y across distributions is wise on my
end. As a reference point here are some numbers from Arch:
 - python2 - 71MiB
 - python3 - 102MiB
 - perl - 50MiB
 - automake - 1.5MiB
 - autoconf - 2 MiB

That said, let's not dwell into this. Size and specifics of each one
is highly distro specific, not to mention that most of those things
(what to bundle, how, etc.) are governed by distribution policies and
decisions.
Since a fair few of those range from philosophical to personal
reasons, it's something I try to stay aware from, whenever possible
;-)

Thus trying to understand why X or Y do not want/like/etc to use tools
A, B or C is a going to end up... yeah.

>> Latter feature seems to be missing from meson afaict, alongside
>> cscope, ctags and a few others that some people still use (I don't
>> personally).
>
> Honestly, I'd be happy to burn distribution tarballs and just use 'git
> evtag' or similar; https://github.com/cgwalters/git-evtag has great
> discussion on the problems that reliance on tarballs causes for people
> trying to do reproducible builds. But that's a discussion for another
> day ...
>
>From a quick look - both permalink and comments are broken (not found) :-(
I'll search it up at some point, thanks.

> I didn't even realise Automake had native cscope support; for me it's
> not particularly useful since I do srcdir != builddir builds, and
> placing my editor root in the builddir seems very odd, but ...
>
>> Thus do check with distro maintainers/others who ship wayland/weston,
>> _if_ in the future one decides to drop the autohell setup ;-)
>
> I'd say anyone doing cross-compilation would be very happy, since with
> the attached script, all I needed to do to make cross-compiling work,
> including generating a build-native wayland-scanner binary, was to add
> 'native: true' to a couple of places and use the below
> cross-compilation definition file; with Autotools, you end up doing
> two separate builds, or insane gymnastics.
>
>> Please consider using a low traffic medium (hint not wayland-devel)
>> since the question _will_ get lost.
>
> Yeah, I'd be fine with pinging them all individually. Like I say, a
> long-ish transition period to get everyone adjusted and make sure we
> don't have any issues with downstreams or individual users, is
> something we absolutely need. I don't want to get rid of Autotools
> overnight for everyone, but I do want to only ever have to use it
> again myself when I change the build, to make sure I haven't broken
> it. :)
>
Ack, makes sense.

As always, thanks for the educational input.

-Emil