[cairo] Got make check to work using CAIRO_REF_DIR, for image target

darxus at chaosreigns.com darxus at chaosreigns.com
Thu Jun 20 10:42:40 PDT 2013

Much thanks to Uli Schlachter and Chris Wilson for getting this to work.
The final step was this bug:

This is how I got it to work:

Xvfb :1 &

/usr/bin/time ./autogen.sh --prefix=$HOME/install.cairo # 0:19.74elapsed

/usr/bin/time make # 0:08.66elapsed

DISPLAY=:1 CAIRO_TEST_TARGET=image /usr/bin/time make test  # this will fail, 1:04.28elapsed
# 457 Passed, 44 Failed [0 crashed, 13 expected], 31 Skipped
#             ^^ Note the number that fail, make sure this changes next
#                time, or you got CAIRO_REF_DIR wrong.

cp -a test/output/ $HOME/source/cairo-ref.1.xvfb

DISPLAY=:1 CAIRO_REF_DIR=$HOME/source/cairo-ref.1.xvfb CAIRO_TEST_TARGET=image /usr/bin/time make test
# 499 Passed, 2 Failed [0 crashed, 2 expected], 31 Skipped
# PASS: cairo-test-suite

# You might want these later:
git log -1 --pretty=format:"%H" > $HOME/source/cairo-ref.1.xvfb/commit.txt
cp -a test/cairo-test-suite.log $HOME/source/cairo-ref.1.xvfb 

The contents of my CAIRO_REF_DIR:

This was done with the current git master, commit
b7331f0c52cc64f2c224eac502afa6c50a1a8d8b .

image is only one of many cairo test targets.  It's just the one that
seemed simplest to get working.  

/usr/bin/time can be omitted everywhere, it just tells you how long
something took to run.

Getting cairo's make test to pass against the existing reference images is
hard.  The test/README documents how you can run make test, copy the output
images to a new reference directory, make your changes, and then run make
test again against your own reference images, using CAIRO_REF_DIR.

Xvfb is a software X server recommended in test/README because apparently X
server driver differences cause problems.

Although after I ran the above, I ran make test again with DISPLAY=:0,
which is a normal X server with (open source) radeon drivers, and again
it passed.  I guess that's cool?  I don't actually know if the image target
hits anything where driver differences could be relevant.

I verified this works from cron (automated testing is awesome), with
this script:

git clean -xfd
./autogen.sh --prefix=$HOME/install.cairo --enable-gl --enable-xcb --disable-ps --disable-script || exit 125
make || exit 1
DISPLAY=:1 CAIRO_REF_DIR=$HOME/source/cairo-ref.1.xvfb CAIRO_TEST_TARGET=image make test

And this cron job:

0 4 * * * cd $HOME/source/cairo && $HOME/source/configmakecheck.cairo.sh > $HOME/source/cairo.cron.log 2>&1

Should some of these reference images get updated?

"All that is necessary for evil to triumph is for good men to do nothing"
- War and Peace (film series)

More information about the cairo mailing list