[cairo] Unit testing? (PDF)

[Chris Wilson]

On Fri, 2009-05-22 at 13:46 -0300, Ian Britten wrote:
> - Given that the Cairo unit tests basically just:
>         printf("Please check %s to ensure it looks/prints correctly")
>    I'm guessing there may not be an easy way to do this.  :(

Yes, a few PDF tests have not been automated. These are generally ones
that are not testing rendering exactness, but other PDF features.

>    Is there a non-easy way?  :P

The methodology for the cairo-test-suite is to run a test case against a
pdf-surface, first do a byte-compare of the vector output (.pdf) against
a known fail/pass and then, for ambiguous cases, rasterise the .pdf
(using poppler, see any2ppm for an example of to use poppler/cairo) and
compare the image against a reference.

At the moment, you do not have full control over the metadata included
with the pdf-surface (such as creation date, creator [cairo-1.9.1]) so
using a byte compare for external unit testing is slightly more
complicated. Another wishlist item (that is on my todo list) is also to
disable the compression within the pdf, which would make diffs against a
reference file more readable.

However, I think the semantic content of the pdf is far more important
to be checking in your unit tests. Does the pdf render correctly? Does
it include the correct text? Is that text selectable? Are the images
embedded correctly? For these poppler has a some tools to extract
information, as well as pdftk .

For unit testing of your own code, I would advise not to use cairo at
all and use a stub library that traced the calls to cairo instead -
though cairo-trace maybe useful here. (Adding the external dependency
upon cairo for your integration and higher level testing.) If you have a
unit test that is not covered by the cairo test suite, then that is a
severe bug in our test suite and we would like to know!

Have fun testing Cairo!
-ickle