[cairo] Unit testing? (PDF)

[Ian Britten]

Chris Wilson wrote:

> The methodology for the cairo-test-suite is to run a test case against a
> pdf-surface, first do a byte-compare of the vector output (.pdf) against
> a known fail/pass and then, for ambiguous cases, rasterise the .pdf
> (using poppler, see any2ppm for an example of to use poppler/cairo) and
> compare the image against a reference.

Could you clarify how there'd be any ambiguity?  Wouldn't the comparison
always fail, or always pass, depending on what meta-data was in the
file, and whether it changes each run?
Or, could someone clarify what meta-data is being written out to the
PDF files?  I see that some (All?) of the PS metadata is documented
http://www.cairographics.org/manual/cairo-ps-surface.html#cairo-ps-surface-dsc-comment
but I'm unsure what is in the PDF file (And I am not sure how to tell).

> However, I think the semantic content of the pdf is far more important
> to be checking in your unit tests. Does the pdf render correctly? 

Yes, this is what I'm mostly interested in.  Users primary concern is
whether the output "looks right", and that is what I'm (initially)
trying to get tests for.

> For unit testing of your own code, I would advise not to use cairo at
> all and use a stub library that traced the calls to cairo instead -
> though cairo-trace maybe useful here. (Adding the external dependency
> upon cairo for your integration and higher level testing.)

Hun?  Not use cairo?  Maybe I wasn't clear (Or misunderstood you)...
I already have a large suite of tests for our various parts, including
the data retrieval, rendering engine, etc.  We render to various
abstract outputs (Canvases), now including Cairo.  As such, it is the
bridge code between our rendering code and Cairo that I'm looking to
test:
  - Did the transformations get set up in Cairo correctly?
  - Did the cap style get correctly set in Cairo for the line?
  - etc, etc.

Obviously, testing this code quickly gets hard when the Cairo surface
we're drawing to could be PDF/PNG/SVG/etc, and the users want each
result to "look right"...  Thus, I was focusing on just one format
for now.

Hmmm...  I was just thinking that maybe what I want for general testing
of this code is some sort of mock/proxy/testing Surface - Is a Surface
hard to make?
Or, is that what cairo-trace is about?  Are there any details around
about it?  I'm not familiar with it... (And, is it in 1.8.x?)

> If you have a unit test that is not covered by the cairo test suite,
 > then that is a severe bug in our test suite and we would like to know!

Absolutely!  However, I'm mostly looking to test our bridge code, to
ensure that settings are being propagated to Cairo correctly.  Rest
assured that if any Cairo problems are found, I'd bring them to your
attention (At which point, you can extend your test suite...)

> Have fun testing Cairo!
[ Well, I'm not looking to test _Cairo_ per se...  ;)  ]

Many thanks!
Ian