[Mesa-dev] [RFC libdrm 0/2] Replace the build system with meson

Tue Mar 21 18:56:49 UTC 2017

On 21 March 2017 at 18:06, Matt Turner <mattst88 at gmail.com> wrote:
> On Tue, Mar 21, 2017 at 10:16 AM, Emil Velikov <emil.l.velikov at gmail.com> wrote:
>> On 21 March 2017 at 15:57, Matt Turner <mattst88 at gmail.com> wrote:
>>> On Mon, Mar 20, 2017 at 12:39 PM, Emil Velikov <emil.l.velikov at gmail.com> wrote:
>>>> On 20 March 2017 at 18:30, Matt Turner <mattst88 at gmail.com> wrote:
>>>>> On Mon, Mar 20, 2017 at 6:55 AM, Emil Velikov <emil.l.velikov at gmail.com> wrote:
>>>>>> Seems like we ended up all over the place, so let me try afresh.
>>>>>>
>>>>>> Above all:
>>>>>>  - Saying "I don't care" about your users is arrogant - let us _not_
>>>>>> do that, please ?
>>>>>
>>>>> Let's be honest, the OpenBSD is subjecting itself to some pretty
>>>>> arbitrary restrictions caused including Mesa in its core: 10+ year old
>>>>> GCC,
>>>> IIRC Brian was using old MinGW GCC, which was one of the blockers - it
>>>> wasn't OpenBSD to blame here ;-)
>>>>
>>>>> non-GNU Make, and now no Meson. I don't believe either FreeBSD or
>>>>> NetBSD keep Mesa as part of the core operating system, and as such
>>>>> don't suffer from these problems.
>>>>>
>>>>> For better or worse, they have made their choices and they get to live
>>>>> with them. We are not beholden to them.
>>>>>
>>>> Overall this hunk seems misplaced. I go agree that we should not cater
>>>> for all their needs. At the same time, intentionally, breaking things
>>>> while we all can coexist is very strange.
>>>>
>>>>>> Even Linux distribution maintainers have responded that "upstream does
>>>>>> not care us", which is indicative that we should be more careful what
>>>>>> we say.
>>>>>
>>>>> Citation needed.
>>>>>
>>>> https://bugs.freedesktop.org/show_bug.cgi?id=98487#c4 and there's a
>>>> couple of instances where I've been contacted in private.
>>>>
>>>>>> For the rest - we're dealing with two orthogonal issues here:
>>>>>>
>>>>>> * Multiple build systems
>>>>>> I believe we'll all agree that I might be the person who's been in all
>>>>>> the build systems the most.
>>>>>> Yes I _would_ _love_ to drop it all but we simply _cannot_ do that yet:
>>>>>
>>>>> No one is advocating dropping all of the existing build systems yet.
>>>>>
>>>>> This patch is an RFC for a smaller project to start the discussion about Mesa.
>>>>>
>>>>>>  - [currently] there is no viable solution for Android
>>>>>
>>>>> Acknowledged. Dylan is going to see if this is something that can be
>>>>> solved in upstream Meson.
>>>>>
>>>> I would suggest checking with Android people as well, as Meson.
>>>> There's some plans on moving to yet another build system - Blueprint.
>>>>
>>>>>>  - dropping the Autotools will lead to OpenBSD and NetBSD having to
>>>>>> write one from scratch, IIRC Solaris/FreeBSD and others are in similar
>>>>>> boat.
>>>>>
>>>>> Solaris is a closed source operating system whose developers do not
>>>>> contribute to the project. We do not need to base our decisions on
>>>>> them.
>>>>>
>>>>> Mesa is not subject to ridiculous requirements (like in the case of
>>>>> OpenBSD) in FreeBSD and NetBSD.
>>>> Again - not a requirement, but coexistence.
>>>>
>>>>> Mesa depends on gmake in FreeBSD's
>>>>> ports, for instance.
>>>>>
>>>> That amongst others is a bug in their packaging.
>>>
>>> That is not even remotely the point.
>>>
>>> My point is that they are not subjecting themselves (and us by
>>> extension, since you want to let them) to such ridiculous
>>> requirements.
>>>
>> I will kindly ask you to keep these adjectives outside of what aims to
>> be (?) technical discussion.
>
> Huh?
>
> Didn't you call us arrogant in this very thread? Didn't you suggest
> we're immature?
>
Should I was unclear previously - I'm not trying to belittle, insult
or otherwise offend your/anyone's contribution or input.

I'm hoping to have a technical discussion, which does not have phrases
such as "slow as mud" "i don't care", "X is ridiculous" and friends.

>> And on your question - 50-100 lines worth of compatibility changes
>> across a 6.5kloc of build system is not that unreasonable, right ?
>
> You're still not understanding my point.
>
>>>>> So I don't think any of this is true.
>>>>>
>>>> I'd suggest giving it a try then - grab a non-GNU make, a release
>>>> tarball and let me know if you spot any issues.
>>>>
>>>>>> These projects have been getting closer to upstream and "forcing" the
>>>>>> extra obstacle is effectively giving them the middle finger.
>>>>>
>>>>> No. Requiring us to compile with a 10 year old GCC is giving a middle finger.
>>>>>
>>>> Can we stop with the GCC thing ? Can we point to a place where we want
>>>> to use feature A introduced with GCC B and we don't ?
>>>
>>> Are you freaking serious?!
>>>
>>> This happens *all* the time. It happened like two days ago with commit
>>> 28b134c75c1fa3b2aaa00dc168f0eca35ccd346d. I'm sure it probably
>>> happened at least once in the previous month, and every month before
>>> that.
>>>
>> Considering none of the ANV code is built outside of Linux, why you
>> guys will restrict yourself is beyond me.
>
> I think you're confused.
>
Can you elaborate ?

You pointed out that we're restricting ourselves to old GCC version
due to OpenBSD. Yet the example given is not code that is build, or
expected to build soon on said platform.
Hence, If you guys want to use functionality from GCC 4.7 [which I
would encourage you to do] it would be better to add a 3 line
configure check - just like st/nine and st/clover already do.

>> Nine requires GCC 4.6 and Clover GCC 4.7 for a long time.
>>
>>>> The anonymous unions/structs for example require newer GCC and we use
>>>> them extensively. If anything we have the workaround(s) for MSVC's
>>>> lack of designated initializers.
>>>
>>> We actually have
>>>
>>>     if test "x$USE_GNU99" = xyes; then
>>>         CFLAGS="$CFLAGS -std=gnu99"
>>>     else
>>>         CFLAGS="$CFLAGS -std=c99"
>>>     fi
>>>
>>> in configure.ac to work around gcc-4.4 incompatibilities.
>>>
>> 5-10 lines of configure code is not that much of a burden, right ?
>
> You're missing the point again.
>
>>>> Not to mention that some/many of the restrictions are imposed by very
>>>> older enterprise linuxes.
>>>
>>> which eventually go out of support and those requirements disappear.
>>> Unlike OpenBSD's insistence on using the last GPLv2 GCC.
>>>
>> At which point we can reconsider, right - perhaps they have the
>> functionality backported, have moved to new version/another compiler
>> etc.
>
> You're missing the point again.
>
Can you/anyone elaborate on this/other points that I'm missing ? Please ?

>>>>>> * Slow build times
>>>>>> Before we jump into "the next cool thing", let us properly utilise
>>>>>> what we have at the moment.
>>>>>
>>>>> It cannot be properly utilized. There is a patch on the list *today*
>>>>> that is attempting to workaround the *design* of libtool. It's an
>>>>> issue that's existed... since libtool?
>>>>>
>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=58664
>>>>> https://lists.gnu.org/archive/html/libtool/2003-06/msg00068.html
>>>>>
>>>> Yes design is ..., but I'm talking about utilising all the X cores on $platform.
>>>>
>>>>>>  - I've asked multiple times about numbers behind those "let's make
>>>>>> the build faster" patches, but never got any :-\
>>>>>
>>>>> What patches? The patches in this thread? What is your question?
>>>>>
>>>> Nearly every time we had a "let's remove this recursive Makefile"
>>>> patch I asked "what improvement are we talking about here - how long
>>>> does it take before/after this patch to build mesa".
>>>> I'm yet to see any numbers :-\
>>>
>>> Because it's basically always small.
>> IIRC Jason, mentioned something similar to "loads" with the recent ANV
>> move - silly me cannot find the quote :-\
>>
>> Found another one though (from the glsl/nir move to compiler/)
>>
>> On 16 December 2015 at 20:24, Matt Turner <mattst88 at gmail.com> wrote:
>>> On Wed, Dec 16, 2015 at 8:53 AM, Emil Velikov <emil.l.velikov at gmail.com> wrote:
>>
>>> > Can anyone confirm how much of a penalty are we talking about (single
>>> > vs separate makefiles) ?
>>>
>>> I'm not going to take the time to quantify it. Just do a clean build
>>> and watch as 7 of your 8 cores sit idle as format_srgb.c is generated
>>> or shared-glapi/libglapi.la is linked. (A dual-core system is not
>>> going to demonstrate this issue properly)
>>
>> This is where I should have said - "I don't have a 8 core system, so I
>> cannot measure it", although on your end one could have let it compile
>> [say, during your lunch] and share with some numbers.
>> Your reply is quite provocative and even if I've had an full-retard
>> moment, I'm not sure that this is the way to reply.
>>
>>> The whole project needs to be
>>> non-recursive, otherwise you get lots of little directories and stalls
>>> (generating format_srgb comes to mind). All the work making things
>>> non-recursive are opportunistic, and don't really address the
>>> completely intractable nature of the problem: there are still 95
>>> Makefile.ams.
>>>
>> Fully agree - those can be reworked, but I simply cannot measure any
>> difference on the systems I have access to.
>> Hence, cannot estimate the severity of the problem.
>>
>>>> Some cases that come to mind - mapi (gles1/2), glsl, nir and the latest ANV.
>>>>
>>>>>>  - I can improve things but would need access to a fancy XX core
>>>>>> system to do rudimentary benchmarks/checks and test patches.
>>>>>
>>>>> Have you ever seen an autotools build system for a project as complex
>>>>> as Mesa that is both non-recursive and comprehensible? I have not, and
>>>>> I did a lot of searching. In my opinion, this is an intractable
>>>>> problem.
>>>>>
>>>> Haven't looked at all really - off the top of my head openvswitch comes to mind.
>>>> Then again, It's not as extensive as mesa :-\
>>>
>>> Just looked -- you know what their last commit to Makefile.am is?
>>>
>>>     Makefile: Drop vestiges of support for non-GNU Make.
>>>
>> That's their decision. I am pointing out that one can have a sane
>> project w/o recursive makefiles, yet perfectly readable.
>>
>>>>> ... and with Meson it is tractable. And it doesn't use libtool. And it
>>>>> can replace at least 2 and maybe all three of our build systems.
>>>>>
>>>>> Those all seem extremely compelling to me, and I think I've done
>>>>> enough work on Mesa's build system and on a downstream distribution to
>>>>> have a valuable opinion on the matter.
>>>> Yes you did a lot of work on the autotools side, with less so on scons
>>>> and android.
>>>>
>>>> What I'm saying is - let us be mature and stop it with the [reasonable
>>>> or not] hatred towards autotools.
>>>
>>> Troll bait.
>>>
>> ? You're saying that nobody is bashing on autotools/friends... ok.
>
> They are not my favorite pieces of software and I think there are
> sound technical arguments in favor of Meson in comparison to them,
> which I and others have made repeatedly in this thread.
>
> None of that is immature.
>
Disagreeing with something/someone is fine, calling it names seems
immature... at least for me.

>>>> It is far from perfect, yet we can improve things on our end rather
>>>> than throwing everything in the bin.
>>>
>>> I have. Again, I think I have done enough of that that I have some
>>> credibility on the matter.
>>
>> Not trying to belittle any of your work, but this does not parse in
>> reply to my suggestion, sorry.
>>
>>
>> Overall your replies seem quite spontaneous/emotional. If I'm making
>> you upset - I do apologise.
>> I'm simply trying to get as much technical details, which I seems to
>> be tailing at.
>
> Let me try one last time:
>
> (1) Non-recursive automake is necessary for parallel build performance
Fully agree

> (2) Non-recursive automake is intractably unmaintainable for
> sufficiently large projects
Not sure I agree here. Do the src/intel/Makefile* files, seem unmaintainable ?

> (3) Mesa is a sufficiently large project
>
Again - fully agree.

> Therefore using automake will be either bad for parallel build
> performance, unmaintainable, or both.
>
> Meson aims to be a build system actually capable of replacing
> autotools (IMO unlike cmake, scons, waf, et al.). It offers a much
> cleaner domain specific language for writing the build rules, while
> generating non-recursive ninja build files. It does not use libtool.
> It supports Windows. It supports cross compilation.
>
Does it support, Darwin/MacOSX, Cygwin, any of the BSDs, Solaris (and
alike) platforms,
How about Android - Android.mk or Blueprint.
Does it have "dist", "check", "distcheck" or less commonly used
"ctags" "cscope"  targets ?

> And it has momentum: libepoxy already has a Meson build system. Others
> in the X.Org community are experimenting with it for libinput, Wayland
> and Weston, and the xserver.
>
> All of that makes Meson very compelling.
That's the thing - I'm never said that it's _not_ a very compelling project.
I'm saying that it's not there yet - mostly due to the list above.

Thanks
Emil