Baselining EXA quality (r100)

[Wang Zhenyu]

On 2007.05.16 11:03:38 +0000, Carl Worth wrote:
> In concert with the effort I recently started to baseline EXA's
> performance, I also want to baseline its quality. Again, I did this
> with the hardware I had readily available, (still an r100---haven't
> gotten a fancy new Intel GM965 yet).

Hope you'll get it soon. :)

> 
> The three things I decided to use for testing are the X test suite,
> the rendercheck program, and cairo's test suite. The results I got for
> each are detailed below.
> 
> To summarize the results:
> 
> X test suite: EXA fails fewer tests than XAA (82 compared to 96), but
> 	      I don't know how to interpret details of the failures.

Our QA team should be running xtest regularly, but most tests were on
XAA, that intel driver has no render accel support, we should move more 
effort to EXA in case they have more bandwidth..
A recent bug on xtest was reported http://bugs.freedesktop.org/show_bug.cgi?id=10914
for xserver master. 

> 
> Rendercheck: XAA passes all tests I ran while EXA fails two,
>              (transformed source and mask).
> 
> 	     XAA fails other tests which I did not run to completion,
> 	     (and I haven't run against EXA at all).

Transformed src has limit support in 965 exa code for rotation, and
no transformed mask support, as I should first figure out a clean method
to handle Gen4 programs, and adding configuable parameters so setup and
fragment program can take care. And I don't know if there's any real users
of that. I plan to fall back it for now on 965. 

> 
> Cairo test suite: From a first look, it appears this suite found 1 bug
>                   in XAA and 2 or 3 bugs in EXA. This suite provides
>                   images showing the failures:
> 
> 	http://people.freedesktop.org/~cworth/cairo-exa-vs-xaa/quality/
> 
> Hopefully that's helpful, and hopefully the details below provide
> enough information for anybody who wants to replicate this kind of
> testing with other driver+hardware combinations.
> 

Nice, I'll try to run cairo's test suite. Thanks.