[cairo] win32 build

[Owen Taylor]

On Sat, 2005-06-25 at 10:35 -0700, Carl Worth wrote:
> On Sat, 25 Jun 2005 13:00:12 -0400, Owen Taylor wrote:
> > This makes the process of adding new regression tests pretty
> > annoying for everyone, which isn't really what we want.
> 
> If we're going to have different output on different backends then
> this seems quite unavoidable to me.
> 
> It would be trivial to make cairo_test first look for
> <test>-<backend>-ref.png before <test>-ref.png.

*If* we can get down to one set of reference images per backend, yes,
I'd agree that something like that is useful.

> > > > And certainly for glitz, and potentially for Win32 as well, we may
> > > > not get exact-bit matches with the *same* image across machines.
> [...]
> > I'm already concerned about the Xlib failures on most servers.
> 
> I think the answer here is simply to make the testing code consistent
> with the rendering guarantees we want to advertise. For example, if we
> don't mind a bit or two being different, it's easy to relax the
> bit-for-bit check currently in cairo_test.
>
> Things like text are much trickier of course, but I think we can do
> pretty well by locking things down to a specific, widely available
> font, and turning off hinting, sub-pixel rendering etc.

It's certainly something we can experiment with.

> Finally, I think the only other thing that's really needed is better
> documentation about what to do with the kinds of inter-machine
> failures we're talking about. If users know to look at the results and
> copy the output image to the reference image if it looks "good enough"
> then that should at least get users to the point where they can see
> regressions appear. Some simple scripts to help the user here could
> also help.

We have to distinguish here:

 - Being able maintain a testing environment for Cairo

 - Having 'make check' work out out of the box

I think it's basic rule of packaging that that when you ship a piece
a tarball, 'make check' should have no unexpected failures that you
know about. People rely on 'make check' to tell them whether the
software built correctly. If we don't have that:

 - We make a ton of work for ourselves explaining that, no that
   failure was OK on an older X server.

 - We lose a useful facility: if in Red Hat's "beehive"
   build environment, libpixman miscompiles on s390, having 
   make check fail at build time is *extremely* useful.

I think the default run of tests for 'make check' has to be only tests
that don't require user intervention for custom reference images, or
special X servers, or anything like that.

Regards,
							Owen

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.freedesktop.org/archives/cairo/attachments/20050625/65ebd821/attachment.pgp