Need Some Help Helping Myself

[Josh Triplett]

On Mon, 2008-01-14 at 16:34 -0500, Anthony L. Awtrey wrote:
> I need some help helping myself. I have a performance regression bug
> that I've been told is not a priority for Xorg devs:
> 
> https://bugs.freedesktop.org/show_bug.cgi?id=13647
> 
> Apparently I can use 'git bisect' to isolate where the detrimental
> changes occurred. I'm sure this is a FAQ, but does anyone have docs or a
> HOWTO to start this process? Does anyone have advice on automating
> simple build / test iterations to isolate performance regressions? Any
> advice on the value of profiling tools and techniques? Where in the Xorg
> source should I be looking for changes? The Intel driver? Acceleration
> libs? All of it?

'git bisect' allows you to perform a binary search on a Git repository
to determine which commit resulted in a particular effect.  If you have
Git installed, "man git-bisect" will give you a good overview of the
process.

In order to use it here, you need to know a few things:
      * Which component of Xorg causes the problem?
      * Which versions exhibit the problem?

You have a benchmark which demonstrates the problem, which should help a
lot.  You need some way to clearly classify a given commit as "good" or
"bad"; if you don't apply clear criteria, the bisect will fail.  A
benchmark helps with that.

I suggest starting by testing releases.  Start with Xorg 7.0, and
upgrade individual components until you observe the failure.  Since you
said Xvfb does not exhibit the problem, you might start with components
not used by Xvfb; the Intel driver does seem like a good place to start,
as does anything related to acceleration.  If you must upgrade a set of
components together, that will make the process more difficult.

Do that until you can narrow the problem down to one component; for
example, "Xorg 7.0 with the Xorg 7.2 versions of $VARIOUS_STUFF and $FOO
has the performance regression, but Xorg 7.0 with Xorg 7.2 versions of
only $VARIOUS_STUFF does not.".  Now you know to focus on $FOO.

Go check out the corresponding module from Git, and make sure you can
build it from source against the other components on your system.  Find
the tags corresponding to the releases you tested, and verify that
building and testing them gives the same performance results.

Now, in that repository, run "git bisect bad tag-for-new-version", and
"git bisect good tag-for-old-version".  Git will now give you a version
to test.  Compile and test it; if you get the performance regression,
type "git bisect bad", otherwise type "git bisect good".  Hopefully all
versions will build; if not, you may have to try a nearby revision,
which you can find with "git bisect visualize".

Finally, if all goes well, git bisect will finish by telling you the
exact commit that caused the performance regression.  When this happens,
post the commit SHA1 to the bug report.  If you can, see if you can take
the new version and un-apply the patch; if this works, build and test
the result, and it may not have the performance regression.

Likely ways this may fail:
      * The performance regression occurred as a result of several
        changes, and as a result you can't clearly classify each commit
        as "bad" or "good", so bisect leads you astray.
      * You track everything down to one commit, but that commit says
        "merge huge pile of changes" or similar.  In this case, someone
        still needs to determine what part of the change caused the
        performance regression, but you've still saved them a huge pile
        of work.

Hope this helps,
Josh Triplett