[cairo] Cairo testing for LSB
stewb at linux-foundation.org
Wed Aug 13 17:35:28 PDT 2008
On Wed, 13 Aug 2008, Carl Worth wrote:
> There are various possible issues. One of the first, most essential
> pieces of missing information from you is which backends are failing. A
> single FAIL on a test without saying which backend failed really doesn't
> say much. Newer cairo versions are nice about putting a little report
> together at the end of the run showing a per-test list of which backends
> failed, and a per-backend list of which tests failed.
> Also, the test suite generates a .html file with links to all the
> generated images, (output, reference, and diff). We'll really need to
> see those to be able to say what's going on.
I did go back and add the detail info out of the runs (essentially what is
in the .log file, that I also capture with TET. I didn't find any html
file, but maybe the 1.0.2 tests didn't have this feature or my "port" lost
it. I'll look again at the source tarball.
If I was to move to a newer version of the test suite, with the
additional features, how do you think things will work out testing older
cairo versions? If need be, I can of course filter out tests that would be
expected to fail against >= 1.0.2 (we're held back at this version due to
where the enterprise distros are at).
> Some of the things that can cause lots of "failures" even when things
> are going well include:
> * Differences in details of how X servers (or X server drivers)
> rasterize the graphics
> * Different fonts being used in the tests than in the reference images
> * Differences in details of how the font rasterizer, (freetype, say),
> rasterizes text glyphs.
> We're always working to make the test suite more resilient to these
> issues, (for example, we're bundling more and more of the fonts directly
> with the suite). But it's not something we'll ever be entirely free of.
> We could also do a much better job of documenting exactly what needs to
> be done to be able to get a "perfect" run of the test suite. (And you
> can definitely help us develop such documentation as you go through this
This is great. I can control the process to some degree using our own Xvfb
and fonts. Freetype differences across the distros could become
The Linux Foundation
More information about the cairo