Mesa is painful to bisect, I hope this can be improved

[Thomas “illwieckz” Debesse]

Le 18/10/2024 à 10:47, Michel Dänzer a écrit :
> On 2024-10-18 01:36, Timur Kristóf wrote:
>> Or that you bisect between
>> very old very new versions?
> 
> I suspect the latter.

Yes, it happens that almost every time I have to bisect Mesa, both boundaries of my bisect are “very old” and “very new” enough (it's usually only within a timespan in months)

>> Other than that, I think it's normal and expected that the drivers
>> evolve and some options change.
> 
> As Thomas points out though, such changes potentially cause pain for people who need to bisect across them. So such changes shouldn't be done lightly but only if the benefits clearly outweigh that pain.

It's totally normal and expected things change, now there can be ways to make those changes having less impact on bisecting. Here are some examples I think about, from the most annoying one to the most pleasant one to use.

1. One way to reduce the painful mismatch may also be done by milestoning some of them, but this also have some drawbacks as it requires a pending branch for the sum of the breaking meson changes.

2. For the true/enabled Meson problem, maybe Meson itself can be made more backward-compatible. I noticed some of the pain comes from Meson itself, the same Mesa commit may rebuild with the exact same build options if the build directory is pruned first, without changing any Mesa build option. Having a build directory not rebuildable without pruning it first is a problem I almost never face with CMake so I guess Meson can be improved on that purpose.

3. A third way to make bisecting easier is to do bibisects, basically “binary bisects”. This would solve most of those uses cases.

You can see what The Documentation Foundation does for LibreOffice:
https://wiki.documentfoundation.org/QA/Bibisect

One idea of bibisect implementation is that every now and then a complete build of Mesa would be done and stored in a tarball somewhere, it would be built against an old-enough libc to maximize the cross-distribution compatibility. Then a git repository would commit at the same time some reference of the tarball. That being done, a bibisect tool would just run git bisect over those commits, and a special script would fetch and unpack the binary tarball (that would replace the build step in the bisect process) and provide a way to run any arbitrary command in some environment.

This would make bisecting very straightforward for most uses cases. Once one gets the before/after boundaries of the bibisect, one can finish the bisect by building the few remaining commits from source.

For example the binary tarballs would be done once a week, so the source bisect would only span over a single week of changes. With a single week of changes, chance that Meson options change on every commit is very low.

I myself do a bit of that by keeping some copies of my builds (alongside copies of old build scripts) to make it simpler to check for regressions.

Eero Tamminen also talked about bibisect here:
https://gitlab.freedesktop.org/mesa/mesa/-/issues/12024#note_2611196

> Something that has helped me was having machinery that builds Mesa (among other things) regularly (e.g. daily or weekly), and keeping those builds (+ their date & commit info) around for a year or two.  Build script is in Git, and documents the necessary build options for given time period.
> One can then do quick bisect using LD_LIBRARY_PATH to prebuilt binaries. Git bisect is then needed only for getting the exact commit between the final 2 binary builds, and unlikely to need build option changes.

Something that will be annoying to deal with in setting-up a bibisect infrastructure is that it may be needed to build and ship the related LLVM components as well (llvmpipe, radeonsi, r600 clover…).

Maybe there are other ways I didn't think of that can help to make bisecting easier without preventing Mesa contributors to actually change Mesa as soon as they like. The best solutions are the one that both make bisecting easier and doesn't slow down the merge of breaking changes.

-- 
Thomas “illwieckz” Debesse