[cairo] Cairo testing for LSB

Stew Benedict stewb at linux-foundation.org
Wed Aug 13 17:35:28 PDT 2008

On Wed, 13 Aug 2008, Carl Worth wrote:

> There are various possible issues. One of the first, most essential
> pieces of missing information from you is which backends are failing. A
> single FAIL on a test without saying which backend failed really doesn't
> say much. Newer cairo versions are nice about putting a little report
> together at the end of the run showing a per-test list of which backends
> failed, and a per-backend list of which tests failed.
> Also, the test suite generates a .html file with links to all the
> generated images, (output, reference, and diff). We'll really need to
> see those to be able to say what's going on.

I did go back and add the detail info out of the runs (essentially what is 
in the .log file, that I also capture with TET.  I didn't find any html 
file, but maybe the 1.0.2 tests didn't have this feature or my "port" lost 
it. I'll look again at the source tarball.

If I was to move to a newer version of the test suite, with the 
additional features, how do you think things will work out testing older 
cairo versions? If need be, I can of course filter out tests that would be 
expected to fail against >= 1.0.2 (we're held back at this version due to 
where the enterprise distros are at).

> Some of the things that can cause lots of "failures" even when things
> are going well include:
> * Differences in details of how X servers (or X server drivers)
> rasterize the graphics
> * Different fonts being used in the tests than in the reference images
> * Differences in details of how the font rasterizer, (freetype, say),
> rasterizes text glyphs.
> We're always working to make the test suite more resilient to these
> issues, (for example, we're bundling more and more of the fonts directly
> with the suite). But it's not something we'll ever be entirely free of.
> We could also do a much better job of documenting exactly what needs to
> be done to be able to get a "perfect" run of the test suite. (And you
> can definitely help us develop such documentation as you go through this
> process).

This is great. I can control the process to some degree using our own Xvfb 
and fonts. Freetype differences across the distros could become 

Stew Benedict
The Linux Foundation

More information about the cairo mailing list