[Mesa-dev] [RFC libdrm 0/2] Replace the build system with meson

[Matt Turner]

On Tue, Mar 21, 2017 at 10:16 AM, Emil Velikov <emil.l.velikov at gmail.com> wrote:
> On 21 March 2017 at 15:57, Matt Turner <mattst88 at gmail.com> wrote:
>> On Mon, Mar 20, 2017 at 12:39 PM, Emil Velikov <emil.l.velikov at gmail.com> wrote:
>>> On 20 March 2017 at 18:30, Matt Turner <mattst88 at gmail.com> wrote:
>>>> On Mon, Mar 20, 2017 at 6:55 AM, Emil Velikov <emil.l.velikov at gmail.com> wrote:
>>>>> Seems like we ended up all over the place, so let me try afresh.
>>>>>
>>>>> Above all:
>>>>>  - Saying "I don't care" about your users is arrogant - let us _not_
>>>>> do that, please ?
>>>>
>>>> Let's be honest, the OpenBSD is subjecting itself to some pretty
>>>> arbitrary restrictions caused including Mesa in its core: 10+ year old
>>>> GCC,
>>> IIRC Brian was using old MinGW GCC, which was one of the blockers - it
>>> wasn't OpenBSD to blame here ;-)
>>>
>>>> non-GNU Make, and now no Meson. I don't believe either FreeBSD or
>>>> NetBSD keep Mesa as part of the core operating system, and as such
>>>> don't suffer from these problems.
>>>>
>>>> For better or worse, they have made their choices and they get to live
>>>> with them. We are not beholden to them.
>>>>
>>> Overall this hunk seems misplaced. I go agree that we should not cater
>>> for all their needs. At the same time, intentionally, breaking things
>>> while we all can coexist is very strange.
>>>
>>>>> Even Linux distribution maintainers have responded that "upstream does
>>>>> not care us", which is indicative that we should be more careful what
>>>>> we say.
>>>>
>>>> Citation needed.
>>>>
>>> https://bugs.freedesktop.org/show_bug.cgi?id=98487#c4 and there's a
>>> couple of instances where I've been contacted in private.
>>>
>>>>> For the rest - we're dealing with two orthogonal issues here:
>>>>>
>>>>> * Multiple build systems
>>>>> I believe we'll all agree that I might be the person who's been in all
>>>>> the build systems the most.
>>>>> Yes I _would_ _love_ to drop it all but we simply _cannot_ do that yet:
>>>>
>>>> No one is advocating dropping all of the existing build systems yet.
>>>>
>>>> This patch is an RFC for a smaller project to start the discussion about Mesa.
>>>>
>>>>>  - [currently] there is no viable solution for Android
>>>>
>>>> Acknowledged. Dylan is going to see if this is something that can be
>>>> solved in upstream Meson.
>>>>
>>> I would suggest checking with Android people as well, as Meson.
>>> There's some plans on moving to yet another build system - Blueprint.
>>>
>>>>>  - dropping the Autotools will lead to OpenBSD and NetBSD having to
>>>>> write one from scratch, IIRC Solaris/FreeBSD and others are in similar
>>>>> boat.
>>>>
>>>> Solaris is a closed source operating system whose developers do not
>>>> contribute to the project. We do not need to base our decisions on
>>>> them.
>>>>
>>>> Mesa is not subject to ridiculous requirements (like in the case of
>>>> OpenBSD) in FreeBSD and NetBSD.
>>> Again - not a requirement, but coexistence.
>>>
>>>> Mesa depends on gmake in FreeBSD's
>>>> ports, for instance.
>>>>
>>> That amongst others is a bug in their packaging.
>>
>> That is not even remotely the point.
>>
>> My point is that they are not subjecting themselves (and us by
>> extension, since you want to let them) to such ridiculous
>> requirements.
>>
> I will kindly ask you to keep these adjectives outside of what aims to
> be (?) technical discussion.

Huh?

Didn't you call us arrogant in this very thread? Didn't you suggest
we're immature?

> And on your question - 50-100 lines worth of compatibility changes
> across a 6.5kloc of build system is not that unreasonable, right ?

You're still not understanding my point.

>>>> So I don't think any of this is true.
>>>>
>>> I'd suggest giving it a try then - grab a non-GNU make, a release
>>> tarball and let me know if you spot any issues.
>>>
>>>>> These projects have been getting closer to upstream and "forcing" the
>>>>> extra obstacle is effectively giving them the middle finger.
>>>>
>>>> No. Requiring us to compile with a 10 year old GCC is giving a middle finger.
>>>>
>>> Can we stop with the GCC thing ? Can we point to a place where we want
>>> to use feature A introduced with GCC B and we don't ?
>>
>> Are you freaking serious?!
>>
>> This happens *all* the time. It happened like two days ago with commit
>> 28b134c75c1fa3b2aaa00dc168f0eca35ccd346d. I'm sure it probably
>> happened at least once in the previous month, and every month before
>> that.
>>
> Considering none of the ANV code is built outside of Linux, why you
> guys will restrict yourself is beyond me.

I think you're confused.

> Nine requires GCC 4.6 and Clover GCC 4.7 for a long time.
>
>>> The anonymous unions/structs for example require newer GCC and we use
>>> them extensively. If anything we have the workaround(s) for MSVC's
>>> lack of designated initializers.
>>
>> We actually have
>>
>>     if test "x$USE_GNU99" = xyes; then
>>         CFLAGS="$CFLAGS -std=gnu99"
>>     else
>>         CFLAGS="$CFLAGS -std=c99"
>>     fi
>>
>> in configure.ac to work around gcc-4.4 incompatibilities.
>>
> 5-10 lines of configure code is not that much of a burden, right ?

You're missing the point again.

>>> Not to mention that some/many of the restrictions are imposed by very
>>> older enterprise linuxes.
>>
>> which eventually go out of support and those requirements disappear.
>> Unlike OpenBSD's insistence on using the last GPLv2 GCC.
>>
> At which point we can reconsider, right - perhaps they have the
> functionality backported, have moved to new version/another compiler
> etc.

You're missing the point again.

>>>>> * Slow build times
>>>>> Before we jump into "the next cool thing", let us properly utilise
>>>>> what we have at the moment.
>>>>
>>>> It cannot be properly utilized. There is a patch on the list *today*
>>>> that is attempting to workaround the *design* of libtool. It's an
>>>> issue that's existed... since libtool?
>>>>
>>>> https://bugzilla.redhat.com/show_bug.cgi?id=58664
>>>> https://lists.gnu.org/archive/html/libtool/2003-06/msg00068.html
>>>>
>>> Yes design is ..., but I'm talking about utilising all the X cores on $platform.
>>>
>>>>>  - I've asked multiple times about numbers behind those "let's make
>>>>> the build faster" patches, but never got any :-\
>>>>
>>>> What patches? The patches in this thread? What is your question?
>>>>
>>> Nearly every time we had a "let's remove this recursive Makefile"
>>> patch I asked "what improvement are we talking about here - how long
>>> does it take before/after this patch to build mesa".
>>> I'm yet to see any numbers :-\
>>
>> Because it's basically always small.
> IIRC Jason, mentioned something similar to "loads" with the recent ANV
> move - silly me cannot find the quote :-\
>
> Found another one though (from the glsl/nir move to compiler/)
>
> On 16 December 2015 at 20:24, Matt Turner <mattst88 at gmail.com> wrote:
>> On Wed, Dec 16, 2015 at 8:53 AM, Emil Velikov <emil.l.velikov at gmail.com> wrote:
>
>> > Can anyone confirm how much of a penalty are we talking about (single
>> > vs separate makefiles) ?
>>
>> I'm not going to take the time to quantify it. Just do a clean build
>> and watch as 7 of your 8 cores sit idle as format_srgb.c is generated
>> or shared-glapi/libglapi.la is linked. (A dual-core system is not
>> going to demonstrate this issue properly)
>
> This is where I should have said - "I don't have a 8 core system, so I
> cannot measure it", although on your end one could have let it compile
> [say, during your lunch] and share with some numbers.
> Your reply is quite provocative and even if I've had an full-retard
> moment, I'm not sure that this is the way to reply.
>
>> The whole project needs to be
>> non-recursive, otherwise you get lots of little directories and stalls
>> (generating format_srgb comes to mind). All the work making things
>> non-recursive are opportunistic, and don't really address the
>> completely intractable nature of the problem: there are still 95
>> Makefile.ams.
>>
> Fully agree - those can be reworked, but I simply cannot measure any
> difference on the systems I have access to.
> Hence, cannot estimate the severity of the problem.
>
>>> Some cases that come to mind - mapi (gles1/2), glsl, nir and the latest ANV.
>>>
>>>>>  - I can improve things but would need access to a fancy XX core
>>>>> system to do rudimentary benchmarks/checks and test patches.
>>>>
>>>> Have you ever seen an autotools build system for a project as complex
>>>> as Mesa that is both non-recursive and comprehensible? I have not, and
>>>> I did a lot of searching. In my opinion, this is an intractable
>>>> problem.
>>>>
>>> Haven't looked at all really - off the top of my head openvswitch comes to mind.
>>> Then again, It's not as extensive as mesa :-\
>>
>> Just looked -- you know what their last commit to Makefile.am is?
>>
>>     Makefile: Drop vestiges of support for non-GNU Make.
>>
> That's their decision. I am pointing out that one can have a sane
> project w/o recursive makefiles, yet perfectly readable.
>
>>>> ... and with Meson it is tractable. And it doesn't use libtool. And it
>>>> can replace at least 2 and maybe all three of our build systems.
>>>>
>>>> Those all seem extremely compelling to me, and I think I've done
>>>> enough work on Mesa's build system and on a downstream distribution to
>>>> have a valuable opinion on the matter.
>>> Yes you did a lot of work on the autotools side, with less so on scons
>>> and android.
>>>
>>> What I'm saying is - let us be mature and stop it with the [reasonable
>>> or not] hatred towards autotools.
>>
>> Troll bait.
>>
> ? You're saying that nobody is bashing on autotools/friends... ok.

They are not my favorite pieces of software and I think there are
sound technical arguments in favor of Meson in comparison to them,
which I and others have made repeatedly in this thread.

None of that is immature.

>>> It is far from perfect, yet we can improve things on our end rather
>>> than throwing everything in the bin.
>>
>> I have. Again, I think I have done enough of that that I have some
>> credibility on the matter.
>
> Not trying to belittle any of your work, but this does not parse in
> reply to my suggestion, sorry.
>
>
> Overall your replies seem quite spontaneous/emotional. If I'm making
> you upset - I do apologise.
> I'm simply trying to get as much technical details, which I seems to
> be tailing at.

Let me try one last time:

(1) Non-recursive automake is necessary for parallel build performance
(2) Non-recursive automake is intractably unmaintainable for
sufficiently large projects
(3) Mesa is a sufficiently large project

Therefore using automake will be either bad for parallel build
performance, unmaintainable, or both.

Meson aims to be a build system actually capable of replacing
autotools (IMO unlike cmake, scons, waf, et al.). It offers a much
cleaner domain specific language for writing the build rules, while
generating non-recursive ninja build files. It does not use libtool.
It supports Windows. It supports cross compilation.

And it has momentum: libepoxy already has a Meson build system. Others
in the X.Org community are experimenting with it for libinput, Wayland
and Weston, and the xserver.

All of that makes Meson very compelling.