[Intel-gfx] [ANNOUNCE] xf86-video-intel 2.8.0

[Chris Wilson]

On Fri, 2009-07-31 at 17:25 -0700, Alan W. Irwin wrote:
> I would agree with you if the PTS were proprietary, but it is not.  My sense
> is the PTS provides a framework for any tests users or companies want to put
> in there.  I further agree with you it is likely some bad tests have gotten
> in, but the Intel guy's are free to improve that situation.  They are also
> free to follow your advice and refuse to add decent test to the PTS which
> then perpetuates the current situation which I don't think anybody likes.

After the previous slating intel-gfx took with PTS, I did have a look at
using it as a framework for integrating cairo-perf-trace into. I wasn't
overwhelmed and as I couldn't even fit its dependencies onto my netbook,
it was a non-starter.

The solution I have in mind is a jhbuild buildbot that will poll and
potentially rebuild the entire gfx stack and then run a test suite,
uploading a signed set of results to fd.o. This obviously has the
serious drawback that many variables will change between runs, so it can
only be considered a snapshot of performance in time. However, if we can
even just couple in system profiling it should prove to be a valuable
resource. (Obviously a large amount of work will be involved in
summarising the information, without losing the details.)

The one point that I will contest is the on-going maintenance cost.
There will be a large initial cost to build a useful tool, and adding
each system will need significant manual verification. But unless people
use the system, and thereby find and address problems, the system is
dead. So in order for it to remain relevant it will need continual
investment of time and resources.
-ickle