ANN: SVG_RenderTest 0.1 (was Re: [Clipart] Anyone try librsvg2 on the clipart)

[Bryce Harrington]

On Sat, Jul 09, 2005 at 10:11:21PM +0100, Stephen Silver wrote:
> Jon Phillips wrote:
> > On Sat, 2005-07-09 at 15:02 +0100, Stephen Silver wrote:
> > > We should try to make the SVG files as portable as possible.
> > Did you file a bug on our bug tracker.
> I'm compiling a list of problematic SVG files in release 0.15.
> So far I've found 49.

Okay...  I think it's time to take our SVG QA up a notch.  ;-)

SVG_RenderTest is a new test program for doing regression testing of SVG
rendering.  You can download it from the Inkscape File downloads page,
or directly from here:

    https://sourceforge.net/project/showfiles.php?group_id=93438&package_id=157539

`runtest` is a bash script that takes *.svg files in the current
directory and attempts to render them into png's using Inkscape,
librsvg, cairo, and batik.  It reports PASS or FAIL for each.  )See the
README for the full list of programs that this test expects to be
installed.)

It also performs a visual 'diff' of the rendered images using the
Motiontrack software.  This calculates a score showing how relatively
different the images are, and also generates a PNG that visually
displays the brightness difference of the files.

Here is an example of the results this test generates:

 1.inkscape           PASS :    0 b_w_tiger_susan_park_01
 1.librsvg            PASS :    0 b_w_tiger_susan_park_01
 1.cairo              PASS :    0 b_w_tiger_susan_park_01
 1.batik              PASS :    0 b_w_tiger_susan_park_01
 1.cairo-mdiff        FAIL :    2 b_w_tiger_susan_park_01
 1.inkscape-mdiff     FAIL :    8 b_w_tiger_susan_park_01
 1.librsvg-mdiff      FAIL :    2 b_w_tiger_susan_park_01
 2.inkscape           PASS :    0 daggers
 2.librsvg            PASS :    0 daggers
 2.cairo              PASS :    0 daggers
 2.batik              PASS :    0 daggers
 2.cairo-mdiff        FAIL :    7 daggers
 2.inkscape-mdiff     FAIL :    8 daggers
 2.librsvg-mdiff      FAIL :    7 daggers
 3.inkscape           PASS :    0 sword_hanger
 3.librsvg            PASS :    0 sword_hanger
 3.cairo              PASS :    0 sword_hanger
 3.batik              PASS :    0 sword_hanger
 3.cairo-mdiff        FAIL :    1 sword_hanger
 3.inkscape-mdiff     FAIL :    8 sword_hanger
 3.librsvg-mdiff      FAIL :    1 sword_hanger

The numbers in the third column are either the exit code or, in the case
of mdiff, a score that shows the relative difference between batik's
output and the output from the given program.  The higher the score, the
worse the difference.

You can see examples of the visual diffs here:

    http://www.inkscape.org/svg_rendertest_example/

The motivations for this test are to help:

 1.  Regression testing of known bad SVG's (i.e. for Inkscape QA)
 2.  Compliance testing of Inkscape against the W3C test suite
 3.  SVG compatibility testing between different SVG rendering engines
 4.  Testing of the Open Clip Art Library images (ob OCAL thread)

This first release works well enough on my machine, but certainly leaves
plenty of room for improvement.  ;-)  Please send me patches if you have
ideas for improvements or spot bugs.

Enjoy,
Bryce