[Intel-gfx] [ANNOUNCE] xf86-video-intel 2.8.0

[Chris Wilson]

On Fri, 2009-07-31 at 19:12 -0400, Ben Gamari wrote:
> Testing the fastest chipset would produce useless data --- comparing
> releases on different hardware is stamp-collecting at best and just
> plain misleading at worst. Personally, I would much rather see the Intel
> folks working on actually improving the codebase. Sure, some benchmarking
> is required to do this, but spending developer time on benchmarking just
> for publicity seems like a poor allocation of resources.

Where do you draw the line between benchmarking and performance
profiling? I spend a not inconsiderable amount of time checking Cairo
for performance regressions (because I know that Mozilla do continuous
performance testing and will not accept regressions into their code
base). As such, I feel it is my due diligence to benchmark likely
suspect patches and report those results in the changelogs, and when
soliciting feedback and review on patches. Obviously, I know that this
does not guarantee that there will not be a regression since I've only
performed a very small sampling, but at the very least it catches and
prevents the worst transgressions going upstream. Of course the other
aspect of being strict about avoiding regressions is that we put a lot
of effort into finding ways of speeding up Cairo. And for that we use a
combination of micro-/macro-benchmarks, callgrind and system profilers.
Suggestions for other tools and techniques welcome and very much
appreciated! [The next set of tools I need will be to monitor both the
CPU and GPU (which of course presumes that I can reduce the overhead of
Cairo sufficiently, or find a much faster CPU!). Tracepoints look like
they provide suitable kernel probes, so now I just need some good
visualisation and analysis tools in order to make use of the data.]

So the question I have been asking myself is "just how much more effort
will it be to (a) set up regular performance monitoring of not just
cairo, but the whole gfx stack and (b) to report such benchmarks in both
an accessible manner and sufficient detail for a developer?" And should
we not open our own benchmarking practices and present them for review
and critique?

Never just for publicity.
-ickle