[cairo] Cairo testing for LSB

[Stew Benedict]

On Wed, 13 Aug 2008, Carl Worth wrote:

> Also, the test suite generates a .html file with links to all the
> generated images, (output, reference, and diff). We'll really need to
> see those to be able to say what's going on.
> 
> Some of the things that can cause lots of "failures" even when things
> are going well include:
> 
> * Differences in details of how X servers (or X server drivers)
> rasterize the graphics
> 
> * Different fonts being used in the tests than in the reference images
> 
> * Differences in details of how the font rasterizer, (freetype, say),
> rasterizes text glyphs.
> 
> We're always working to make the test suite more resilient to these
> issues, (for example, we're bundling more and more of the fonts directly
> with the suite). But it's not something we'll ever be entirely free of.
> We could also do a much better job of documenting exactly what needs to
> be done to be able to get a "perfect" run of the test suite. (And you
> can definitely help us develop such documentation as you go through this
> process).
> 

OK, I see the disconnect. It looks like the test framework changed some 
time after 1.0.2. I see the make-html.pl in 1.4.14. I kind of cobbled the 
additional cairo_test_log entries from 1.4.14 into cairo-test.c, but it's 
still not quite working right to generate the html (no boilerplate for 
one thing). I get html, but a bunch of "UNSUPPORTED STATUS (update 
make-html.pl)" because the error text is getting the in the log before the 
"FAIL/UNTESTED", etc). 

Also tried building 1.4.14 tests. I need to build with lsbcc with our 
libcairo.so stub to be portable. Plain "make check-TESTS" works on SLES10, 
but if I try to test against the system libcairo than things fall apart 
(unresolved symbols for interfaces we haven't included in LSB).

Is the additional log data I put on:

http://www.linuxfoundation.org/en/CairoTesting40

(things like Error: 112333 pixels differ from reference image 
./clip-operator-ref.png)

enough to be useful or is there another way I can get at the details you 
need?

Thanks,
-- 
Stew Benedict
The Linux Foundation