Mesa is painful to bisect, I hope this can be improved

[David Heidelberg]

On 12/11/2024 11:38, Thomas “illwieckz” Debesse wrote:
> 
> Le 18/10/2024 à 10:47, Michel Dänzer a écrit :
>> On 2024-10-18 01:36, Timur Kristóf wrote:
>>> Or that you bisect between
>>> very old very new versions?
>>
>> I suspect the latter.
> 
> Yes, it happens that almost every time I have to bisect Mesa, both 
> boundaries of my bisect are “very old” and “very new” enough (it's 
> usually only within a timespan in months)
> 
>>> Other than that, I think it's normal and expected that the drivers
>>> evolve and some options change.
>>
>> As Thomas points out though, such changes potentially cause pain for 
>> people who need to bisect across them. So such changes shouldn't be 
>> done lightly but only if the benefits clearly outweigh that pain.
> 
> It's totally normal and expected things change, now there can be ways to 
> make those changes having less impact on bisecting. Here are some 
> examples I think about, from the most annoying one to the most pleasant 
> one to use.
> 
> 1. One way to reduce the painful mismatch may also be done by 
> milestoning some of them, but this also have some drawbacks as it 
> requires a pending branch for the sum of the breaking meson changes.
> 
> 2. For the true/enabled Meson problem, maybe Meson itself can be made 
> more backward-compatible. I noticed some of the pain comes from Meson 
> itself, the same Mesa commit may rebuild with the exact same build 
> options if the build directory is pruned first, without changing any 
> Mesa build option. Having a build directory not rebuildable without 
> pruning it first is a problem I almost never face with CMake so I guess 
> Meson can be improved on that purpose.
> 
> 3. A third way to make bisecting easier is to do bibisects, basically 
> “binary bisects”. This would solve most of those uses cases.
> 
> You can see what The Documentation Foundation does for LibreOffice:
> https://wiki.documentfoundation.org/QA/Bibisect
> 
> One idea of bibisect implementation is that every now and then a 
> complete build of Mesa would be done and stored in a tarball somewhere, 
> it would be built against an old-enough libc to maximize the cross- 
> distribution compatibility. Then a git repository would commit at the 
> same time some reference of the tarball. That being done, a bibisect 
> tool would just run git bisect over those commits, and a special script 
> would fetch and unpack the binary tarball (that would replace the build 
> step in the bisect process) and provide a way to run any arbitrary 
> command in some environment.
> 

For Debian users, I wanted to mention the Mesa3D nightly repository, 
which I built with bisection in mind.

It's build everyday (except when build starts failing, usually due to 
buildsystem changes and I'm currently only one who maintains it) and you 
can replace the "trixie" (the latest) with hash of previous days (see 
the "History" button) to download older packages.

https://gitlab.freedesktop.org/gfx-ci/ci-deb-repo/-/tree/trixie?ref_type=heads

I believe bibisect is definitely more universal solution, but as someone 
may benefit from this repo, wanted to mention it.

David

> This would make bisecting very straightforward for most uses cases. Once 
> one gets the before/after boundaries of the bibisect, one can finish the 
> bisect by building the few remaining commits from source.
> 
> For example the binary tarballs would be done once a week, so the source 
> bisect would only span over a single week of changes. With a single week 
> of changes, chance that Meson options change on every commit is very low.
> 
> I myself do a bit of that by keeping some copies of my builds (alongside 
> copies of old build scripts) to make it simpler to check for regressions.
> 
> Eero Tamminen also talked about bibisect here:
> https://gitlab.freedesktop.org/mesa/mesa/-/issues/12024#note_2611196
> 
>> Something that has helped me was having machinery that builds Mesa 
>> (among other things) regularly (e.g. daily or weekly), and keeping 
>> those builds (+ their date & commit info) around for a year or two.  
>> Build script is in Git, and documents the necessary build options for 
>> given time period.
>> One can then do quick bisect using LD_LIBRARY_PATH to prebuilt 
>> binaries. Git bisect is then needed only for getting the exact commit 
>> between the final 2 binary builds, and unlikely to need build option 
>> changes.
> 
> Something that will be annoying to deal with in setting-up a bibisect 
> infrastructure is that it may be needed to build and ship the related 
> LLVM components as well (llvmpipe, radeonsi, r600 clover…).
> 
> Maybe there are other ways I didn't think of that can help to make 
> bisecting easier without preventing Mesa contributors to actually change 
> Mesa as soon as they like. The best solutions are the one that both make 
> bisecting easier and doesn't slow down the merge of breaking changes.
>