[cairo] win32 build

[Carl Worth]

On Sat, 25 Jun 2005 13:00:12 -0400, Owen Taylor wrote:
> This makes the process of adding new regression tests pretty
> annoying for everyone, which isn't really what we want.

If we're going to have different output on different backends then
this seems quite unavoidable to me.

It would be trivial to make cairo_test first look for
<test>-<backend>-ref.png before <test>-ref.png.

> > > And certainly for glitz, and potentially for Win32 as well, we may
> > > not get exact-bit matches with the *same* image across machines.
[...]
> I'm already concerned about the Xlib failures on most servers.

I think the answer here is simply to make the testing code consistent
with the rendering guarantees we want to advertise. For example, if we
don't mind a bit or two being different, it's easy to relax the
bit-for-bit check currently in cairo_test.

Things like text are much trickier of course, but I think we can do
pretty well by locking things down to a specific, widely available
font, and turning off hinting, sub-pixel rendering etc.

Finally, I think the only other thing that's really needed is better
documentation about what to do with the kinds of inter-machine
failures we're talking about. If users know to look at the results and
copy the output image to the reference image if it looks "good enough"
then that should at least get users to the point where they can see
regressions appear. Some simple scripts to help the user here could
also help.

-Carl

PS. I've been doing this sort of "visual check" regression testing
with svg2png and the W3C conformance suite since before cairo/test
existed. The script I use in in CVS in cairo/test/testsvg. I use it by
unpacking the W3C suite into cairo/test/svg_suite and from there
running:

	../testsvg *.svg

And if it fails on any images, visually checking the results with an
image viewer and the list of different images that testsvg spits out:

	xg `cat testsvg-imagelist`

And if everything looks fine, copying all the output images over to
reference images:

	cp testsvg-output/* testsvg-reference

(Oh, and the testsvg-reference directory needs to be bootstrapped
originally from testsvg-output. So this only catches future
regressions, not current problems.)

This sort of approach has been necessary since changes in the SVG
output often appear due to changes or improvements in libsvg and
libsvg-cairo that are unrelated to testing cairo itself, which is what
I really care about.

My goal with cairo/test was to get away from manual verification and
I'd still like to do as much as we can in an automated way. But if we
do need to go to something like the above system, I think my
experience argues that it's not actually *that* hard to do.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.freedesktop.org/archives/cairo/attachments/20050625/f740ffe8/attachment.pgp