[cairo] Test suite updates: PDF and PS (Everyone upgrade to gs 8.61!)

[Carl Worth]

I'm still working to get our number of test suite failures down to as
close to 0 as possible. Obviously, part of this is the desire to fix
bugs, and to get to the "all tests passed" state we want to be in for
the release---not to mention that we also want to remove some of the
XFAIL, (== "expected failures"), as well.

Additionally, though, the large number of failures really makes it
hard to get work done. One problem is trying to decide "Did this last
change I made break anything?". If there are a bunch of failures both
before and after the change, how can you tell?

One thing that can help here is the CAIRO_REF_DIR trick. It works like
this:

	# Before code change there may be failures we don't care about
	make test

	# Let's save those output images
	mkdir /some/path/
	cp test/*-out.png /some/path

	# hack, hack, hack

	# Now to see if nothing changed:
	CAIRO_REF_DIR=/some/path make test

That's also a very handy trick if your system just won't generate the
same images as the reference images for some reason, (say you're
running into font rasterization differences for example).

So I've added the above information to cairo/test/README now.

Meanwhile, the other reason I'm actively battling the test suite is
that I want to throw the 16.16->24.8 switch and doing that requires
updating *lots* of reference images. But updating reference images is
no fun at all in the face of failures. So I'm wanting to get as many
failures to go away as possible.

For PDF, what I've done is to run against the latest poppler, as
checked out from git. Then, where pdf tests are failing, but due to
bugs in poppler, (that is, cairo's PDF output looks fine in acroread,
for example), I've been filing poppler bugs, and then just recording
the buggy poppler rendering as the pdf reference image. This makes the
failures go away, and will still alert us if we change the pdf output,
(or if poppler gets updated).

Second, for PostScript, I first updated to ghostscript version 8.61,
and then I regenerated all the reference images with that version of
ghostscript. So anyone that wants to follow along and not have a ton
of cairo-ps failures will want to install and use that same version.

I've also documented this in cairo/test/README.

After all that, we're not quite down to zero test-suite failures
yet. But most[*] of the failures that are left look like actual
bugs. Here's the list I've got now if anyone wants to take a look at
any of these:

	Failures per surface - svg: 2, ps: 4, pdf: 7

	svg:    a8-mask extend-reflect-similar

	ps:     clip-operator degenerate-path miter-precision zero-alpha

	pdf:    clip-operator mask operator-source push-group radial-gradient
		unbounded-operator bitmap-font

Enjoy,

-Carl

[*] I've got one xlib failure (with Xephyr) which I think was
rotate-image-surface-paint, but it didn't look meaningful. Also,
there's a reported failure for operator clear on pdf on the -25 offset
case---but that's just due to poppler not being translationally
invariant in its rendering.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.cairographics.org/archives/cairo/attachments/20080220/635e1ac6/attachment-0001.pgp