Wed Feb 20 06:26:31 PST 2008

dev/null                                       |binary
 test/README                                    |   99 +++++++++++++++++++++----
 test/clip-fill-rule-ps-argb32-ref.png          |binary
 test/clip-fill-rule-ps-rgb24-ref.png           |binary
 test/dash-caps-joins-ps-argb32-ref.png         |binary
 test/dash-caps-joins-ps-rgb24-ref.png          |binary
 test/dash-curve-ps-ref.png                     |binary
 test/dash-state-ps-ref.png                     |binary
 test/dash-zero-length-ps-ref.png               |binary
 test/dash-zero-length-ps-rgb24-ref.png         |binary
 test/degenerate-pen-ps-argb32-ref.png          |binary
 test/font-matrix-translation-ps-argb32-ref.png |binary
 test/font-matrix-translation-ps-rgb24-ref.png  |binary
 test/ft-text-vertical-layout-type1-ps-ref.png  |binary
 test/ft-text-vertical-layout-type3-ps-ref.png  |binary
 test/glyph-cache-pressure-ps-ref.png           |binary
 test/gradient-alpha-ps-argb32-ref.png          |binary
 test/line-width-scale-ps-ref.png               |binary
 test/linear-gradient-ps-ref.png                |binary
 test/linear-gradient-reflect-ps-ref.png        |binary
 test/mask-alpha-svg-argb32-ref.png             |binary
 test/meta-surface-pattern-ps-argb32-ref.png    |binary
 test/meta-surface-pattern-ps-rgb24-ref.png     |binary
 test/new-sub-path-ps-argb32-ref.png            |binary
 test/operator-clear-ps-argb32-ref.png          |binary
 test/operator-source-ps-argb32-ref.png         |binary
 test/operator-source-ps-rgb24-ref.png          |binary
 test/random-intersections-ps-ref.png           |binary
 test/select-font-face-ps-ref.png               |binary
 test/show-text-current-point-ps-ref.png        |binary
 test/text-pattern-ps-argb32-ref.png            |binary
 test/text-pattern-ps-rgb24-ref.png             |binary
 test/trap-clip-ps-argb32-ref.png               |binary
 test/trap-clip-ps-rgb24-ref.png                |binary
 34 files changed, 87 insertions(+), 12 deletions(-)

New commits:
commit f954db72dd33de837296ede353ad9823b0e32fcd
Author: Carl Worth <cworth at cworth.org>
Date:   Wed Feb 20 06:24:48 2008 -0800

    Add svg-specific reference image for mask-alpha
    
    There's no apparent bug here.

diff --git a/test/mask-alpha-svg-argb32-ref.png b/test/mask-alpha-svg-argb32-ref.png
new file mode 100644
index 0000000..3e56aa3
Binary files /dev/null and b/test/mask-alpha-svg-argb32-ref.png differ
commit c197c684a6b002ad8549d92f15d7f68b7c0068bd
Author: Carl Worth <cworth at cworth.org>
Date:   Wed Feb 20 05:58:42 2008 -0800

    Update ps-specific reference images for ghostscript 8.61
    
    I was wrong in the previous message where I said I had ghostscript 8.61
    installed. I do now, (and "gs --version" says the same thing), and
    here are the new reference images.

diff --git a/test/clip-fill-rule-ps-argb32-ref.png b/test/clip-fill-rule-ps-argb32-ref.png
index f06a1a2..67a8988 100644
Binary files a/test/clip-fill-rule-ps-argb32-ref.png and b/test/clip-fill-rule-ps-argb32-ref.png differ
diff --git a/test/clip-fill-rule-ps-rgb24-ref.png b/test/clip-fill-rule-ps-rgb24-ref.png
index 60e43c4..0697a42 100644
Binary files a/test/clip-fill-rule-ps-rgb24-ref.png and b/test/clip-fill-rule-ps-rgb24-ref.png differ
diff --git a/test/dash-caps-joins-ps-argb32-ref.png b/test/dash-caps-joins-ps-argb32-ref.png
index 513fb68..baadcb4 100644
Binary files a/test/dash-caps-joins-ps-argb32-ref.png and b/test/dash-caps-joins-ps-argb32-ref.png differ
diff --git a/test/dash-caps-joins-ps-rgb24-ref.png b/test/dash-caps-joins-ps-rgb24-ref.png
index c62f48d..baadcb4 100644
Binary files a/test/dash-caps-joins-ps-rgb24-ref.png and b/test/dash-caps-joins-ps-rgb24-ref.png differ
diff --git a/test/dash-curve-ps-ref.png b/test/dash-curve-ps-ref.png
index 50f31fb..25a768b 100644
Binary files a/test/dash-curve-ps-ref.png and b/test/dash-curve-ps-ref.png differ
diff --git a/test/dash-state-ps-argb32-ref.png b/test/dash-state-ps-argb32-ref.png
deleted file mode 100644
index 01d32fa..0000000
Binary files a/test/dash-state-ps-argb32-ref.png and /dev/null differ
diff --git a/test/dash-state-ps-ref.png b/test/dash-state-ps-ref.png
new file mode 100644
index 0000000..ec6904e
Binary files /dev/null and b/test/dash-state-ps-ref.png differ
diff --git a/test/dash-state-ps-rgb24-ref.png b/test/dash-state-ps-rgb24-ref.png
deleted file mode 100644
index e655974..0000000
Binary files a/test/dash-state-ps-rgb24-ref.png and /dev/null differ
diff --git a/test/dash-zero-length-ps-argb32-ref.png b/test/dash-zero-length-ps-argb32-ref.png
deleted file mode 100644
index 6c334e1..0000000
Binary files a/test/dash-zero-length-ps-argb32-ref.png and /dev/null differ
diff --git a/test/dash-zero-length-ps-ref.png b/test/dash-zero-length-ps-ref.png
new file mode 100644
index 0000000..68fd3fb
Binary files /dev/null and b/test/dash-zero-length-ps-ref.png differ
diff --git a/test/dash-zero-length-ps-rgb24-ref.png b/test/dash-zero-length-ps-rgb24-ref.png
new file mode 100644
index 0000000..8a4a40f
Binary files /dev/null and b/test/dash-zero-length-ps-rgb24-ref.png differ
diff --git a/test/degenerate-pen-ps-argb32-ref.png b/test/degenerate-pen-ps-argb32-ref.png
index 47b2900..2c23cd4 100644
Binary files a/test/degenerate-pen-ps-argb32-ref.png and b/test/degenerate-pen-ps-argb32-ref.png differ
diff --git a/test/font-matrix-translation-ps-argb32-ref.png b/test/font-matrix-translation-ps-argb32-ref.png
index 9f6a252..b8a9216 100644
Binary files a/test/font-matrix-translation-ps-argb32-ref.png and b/test/font-matrix-translation-ps-argb32-ref.png differ
diff --git a/test/font-matrix-translation-ps-rgb24-ref.png b/test/font-matrix-translation-ps-rgb24-ref.png
index 330a1b5..b8a9216 100644
Binary files a/test/font-matrix-translation-ps-rgb24-ref.png and b/test/font-matrix-translation-ps-rgb24-ref.png differ
diff --git a/test/ft-text-vertical-layout-type1-ps-ref.png b/test/ft-text-vertical-layout-type1-ps-ref.png
index 1329edb..e7929d1 100644
Binary files a/test/ft-text-vertical-layout-type1-ps-ref.png and b/test/ft-text-vertical-layout-type1-ps-ref.png differ
diff --git a/test/ft-text-vertical-layout-type3-ps-ref.png b/test/ft-text-vertical-layout-type3-ps-ref.png
index af1f9dc..7e84d12 100644
Binary files a/test/ft-text-vertical-layout-type3-ps-ref.png and b/test/ft-text-vertical-layout-type3-ps-ref.png differ
diff --git a/test/glyph-cache-pressure-ps-ref.png b/test/glyph-cache-pressure-ps-ref.png
index c8851c6..f0430de 100644
Binary files a/test/glyph-cache-pressure-ps-ref.png and b/test/glyph-cache-pressure-ps-ref.png differ
diff --git a/test/gradient-alpha-ps-argb32-ref.png b/test/gradient-alpha-ps-argb32-ref.png
index 44e25a3..323238b 100644
Binary files a/test/gradient-alpha-ps-argb32-ref.png and b/test/gradient-alpha-ps-argb32-ref.png differ
diff --git a/test/line-width-scale-ps-argb32-ref.png b/test/line-width-scale-ps-argb32-ref.png
deleted file mode 100644
index 4b43302..0000000
Binary files a/test/line-width-scale-ps-argb32-ref.png and /dev/null differ
diff --git a/test/line-width-scale-ps-ref.png b/test/line-width-scale-ps-ref.png
new file mode 100644
index 0000000..9d95aaf
Binary files /dev/null and b/test/line-width-scale-ps-ref.png differ
diff --git a/test/line-width-scale-ps-rgb24-ref.png b/test/line-width-scale-ps-rgb24-ref.png
deleted file mode 100644
index 12c5b52..0000000
Binary files a/test/line-width-scale-ps-rgb24-ref.png and /dev/null differ
diff --git a/test/linear-gradient-ps-argb32-ref.png b/test/linear-gradient-ps-argb32-ref.png
deleted file mode 100644
index e9202d8..0000000
Binary files a/test/linear-gradient-ps-argb32-ref.png and /dev/null differ
diff --git a/test/linear-gradient-ps-ref.png b/test/linear-gradient-ps-ref.png
new file mode 100644
index 0000000..2e0ec89
Binary files /dev/null and b/test/linear-gradient-ps-ref.png differ
diff --git a/test/linear-gradient-ps-rgb24-ref.png b/test/linear-gradient-ps-rgb24-ref.png
deleted file mode 100644
index e9202d8..0000000
Binary files a/test/linear-gradient-ps-rgb24-ref.png and /dev/null differ
diff --git a/test/linear-gradient-reflect-ps-ref.png b/test/linear-gradient-reflect-ps-ref.png
index ddc9197..ea6d25c 100644
Binary files a/test/linear-gradient-reflect-ps-ref.png and b/test/linear-gradient-reflect-ps-ref.png differ
diff --git a/test/meta-surface-pattern-ps-argb32-ref.png b/test/meta-surface-pattern-ps-argb32-ref.png
index 1cd7a80..fbd58ef 100644
Binary files a/test/meta-surface-pattern-ps-argb32-ref.png and b/test/meta-surface-pattern-ps-argb32-ref.png differ
diff --git a/test/meta-surface-pattern-ps-rgb24-ref.png b/test/meta-surface-pattern-ps-rgb24-ref.png
index 0f65148..84da8da 100644
Binary files a/test/meta-surface-pattern-ps-rgb24-ref.png and b/test/meta-surface-pattern-ps-rgb24-ref.png differ
diff --git a/test/new-sub-path-ps-argb32-ref.png b/test/new-sub-path-ps-argb32-ref.png
index 76c50fb..69257d9 100644
Binary files a/test/new-sub-path-ps-argb32-ref.png and b/test/new-sub-path-ps-argb32-ref.png differ
diff --git a/test/operator-clear-ps-argb32-ref.png b/test/operator-clear-ps-argb32-ref.png
index 7840e9b..1cf666f 100644
Binary files a/test/operator-clear-ps-argb32-ref.png and b/test/operator-clear-ps-argb32-ref.png differ
diff --git a/test/operator-source-ps-argb32-ref.png b/test/operator-source-ps-argb32-ref.png
index 0044b74..acb1e37 100644
Binary files a/test/operator-source-ps-argb32-ref.png and b/test/operator-source-ps-argb32-ref.png differ
diff --git a/test/operator-source-ps-rgb24-ref.png b/test/operator-source-ps-rgb24-ref.png
index 60451f6..f330491 100644
Binary files a/test/operator-source-ps-rgb24-ref.png and b/test/operator-source-ps-rgb24-ref.png differ
diff --git a/test/pixman-rotate-ps-argb32-ref.png b/test/pixman-rotate-ps-argb32-ref.png
deleted file mode 100644
index 1d6ad04..0000000
Binary files a/test/pixman-rotate-ps-argb32-ref.png and /dev/null differ
diff --git a/test/pixman-rotate-ps-rgb24-ref.png b/test/pixman-rotate-ps-rgb24-ref.png
deleted file mode 100644
index d8ecce9..0000000
Binary files a/test/pixman-rotate-ps-rgb24-ref.png and /dev/null differ
diff --git a/test/random-intersections-ps-argb32-ref.png b/test/random-intersections-ps-argb32-ref.png
deleted file mode 100644
index 742d30f..0000000
Binary files a/test/random-intersections-ps-argb32-ref.png and /dev/null differ
diff --git a/test/random-intersections-ps-ref.png b/test/random-intersections-ps-ref.png
new file mode 100644
index 0000000..0813685
Binary files /dev/null and b/test/random-intersections-ps-ref.png differ
diff --git a/test/random-intersections-ps-rgb24-ref.png b/test/random-intersections-ps-rgb24-ref.png
deleted file mode 100644
index 742d30f..0000000
Binary files a/test/random-intersections-ps-rgb24-ref.png and /dev/null differ
diff --git a/test/select-font-face-ps-ref.png b/test/select-font-face-ps-ref.png
index 57526cc..c63f076 100644
Binary files a/test/select-font-face-ps-ref.png and b/test/select-font-face-ps-ref.png differ
diff --git a/test/show-text-current-point-ps-ref.png b/test/show-text-current-point-ps-ref.png
index 198d9c6..c8b42e1 100644
Binary files a/test/show-text-current-point-ps-ref.png and b/test/show-text-current-point-ps-ref.png differ
diff --git a/test/text-pattern-ps-argb32-ref.png b/test/text-pattern-ps-argb32-ref.png
index afbc7c4..6a727b9 100644
Binary files a/test/text-pattern-ps-argb32-ref.png and b/test/text-pattern-ps-argb32-ref.png differ
diff --git a/test/text-pattern-ps-rgb24-ref.png b/test/text-pattern-ps-rgb24-ref.png
index d3d58d7..6ed28e4 100644
Binary files a/test/text-pattern-ps-rgb24-ref.png and b/test/text-pattern-ps-rgb24-ref.png differ
diff --git a/test/text-rotate-ps-argb32-ref.png b/test/text-rotate-ps-argb32-ref.png
deleted file mode 100644
index 125965f..0000000
Binary files a/test/text-rotate-ps-argb32-ref.png and /dev/null differ
diff --git a/test/text-rotate-ps-rgb24-ref.png b/test/text-rotate-ps-rgb24-ref.png
deleted file mode 100644
index 125965f..0000000
Binary files a/test/text-rotate-ps-rgb24-ref.png and /dev/null differ
diff --git a/test/trap-clip-ps-argb32-ref.png b/test/trap-clip-ps-argb32-ref.png
index b6677a6..8bbd1fd 100644
Binary files a/test/trap-clip-ps-argb32-ref.png and b/test/trap-clip-ps-argb32-ref.png differ
diff --git a/test/trap-clip-ps-rgb24-ref.png b/test/trap-clip-ps-rgb24-ref.png
index 0c692d2..f3ded80 100644
Binary files a/test/trap-clip-ps-rgb24-ref.png and b/test/trap-clip-ps-rgb24-ref.png differ
commit 8d2d4130146d16676c9ed6cf7d82d6aeaadf1c56
Author: Carl Worth <cworth at cworth.org>
Date:   Wed Feb 20 05:11:53 2008 -0800

    Expand the documentation of cairo/test/README
    
    Add notes on CAIRO_REF_DIR and the version requirements of
    poppler and ghostscript necessary for getting no failures.

diff --git a/test/README b/test/README
index 988177d..341a795 100644
--- a/test/README
+++ b/test/README
@@ -1,8 +1,10 @@
 Regression test suite for cairo.
 
+How to use cairo's test suite
+=============================
 Using this test should be as simple as running:
 
-	make check
+	make test
 
 assuming that the cairo distribution in the directory above has been
 configured and built. The test suite here goes through some effort to
@@ -10,14 +12,17 @@ run against the locally compiled library rather than any installed
 version, but those efforts may fall short depending on the level of your
 libtool madness.
 
-The test suite needs to be run before any code is committed and before
-any release. Here are the rules governing the use of the suite:
+The results of the test suite run are summarized in an index.html
+file, which, when viewed in a web browser makes it quite easy to
+visually see any failed renderings alongside the corresponding
+reference image, (and a diff image as well).
 
+The test suite needs to be run before any code is committed and before
+any release. See below for hints and rules governing the use of the suite:
 
 Tailoring tests running
 -----------------------
-
-There are some mechanisms to limit the tests run during "make check".
+There are some mechanisms to limit the tests run during "make test".
 These come very handy when doing development, but should not be used
 to circumvent the "pass" requirements listed below.
 
@@ -27,12 +32,86 @@ It should contain a (space-, comma-, etc-separated) list of backends to test.
 To limit the tests run, use the TESTS make variable, which should be a
 space-separated list of tests to run.  For example:
 
-  make check TARGETS=image,ps TESTS="zero-alpha"
+  make test TARGETS=image,ps TESTS="zero-alpha"
+
+Another very handy mechanism when trying to fix bugs is:
+
+  make retest
+
+This will re-run the test suite, but only on tests that failed on the
+last run. So this is a much faster way of checking if changes actually
+fix bugs rather than running the entire test suite again.
+
+Getting the elusive zero failures
+---------------------------------
+It's generally been very difficult to achieve a test run with zero
+failures. The difficulties stem from the various versions of the many
+libraries that the test suite depends on, (it depends on a lot more
+than cairo itself), as well as fonts and other system-specific
+settings. If your system differs significantly from the system on
+which the reference images were generated, then you will likely see
+the test suite reporting "failures", (even if cairo is working just
+fine).
+
+We are constantly working to reduce the number of variables that need
+to be tweaked to get a clean run, (for example, by bundling fonts with
+the test suite itself), and also working to more carefully document
+the software configuration used to generate the reference images.
+
+Here are some of the relevant details:
+
+  * Your system must have a copy of the Bitstream Vera font, (this is
+    a free software font that is often in a file named Vera.ttf).
+
+  * Currently, you must be using a build of cairo using freetype
+    (cairo-ft) as the default font backend. Otherwise all tests
+    involving text are likely to fail.
+
+  * To test the pdf backend, you will want the very latest version of
+    poppler as made available via git:
+
+	git clone git://anongit.freedesktop.org/git/poppler/poppler
 
+    As of this writing, no released version of poppler contains all
+    the fixes you will need to avoid false negatives from the test
+    suite.
+
+  * To test the ps backend, you will need ghostscript version 8.61.
+
+What if I can't make my system match?
+-------------------------------------
+For one reason or another, you may be unable to get a clean run of the
+test suite even if cairo is working properly, (for example, you might
+be on a system without freetype). In this case, it's still useful to
+be able to determine if code changes you make to cairo result in any
+regressions to the test suite. But it's hard to notice regressions if
+there are many failures both before and after your changes.
+
+For this scenario, you can capture the output of a run of the test
+suite before your changes, and then use the CAIRO_REF_DIR environment
+variable to use that output as the reference images for a run after
+your changes. The process looks like this:
+
+        # Before code change there may be failures we don't care about
+        make test
+
+        # Let's save those output images
+        mkdir /some/directory/
+        cp test/*-out.png /some/directory/
+
+        # hack, hack, hack
+
+        # Now to see if nothing changed:
+        CAIRO_REF_DIR=/some/directory/ make test
+
+Best practices for cairo developers
+===================================
+If we all follow the guidelines below, then both the test suite and
+cairo itself will stay much healthier, and we'll all have a lot more
+fun hacking on cairo.
 
 Before committing
 -----------------
-
 All tests should return a result of PASS or XFAIL. The XFAIL results
 indicate known bugs. The final message should be one of the following:
 
@@ -42,7 +121,6 @@ indicate known bugs. The final message should be one of the following:
 If any tests have a status of FAIL, then the new code has caused a
 regression error which should be fixed before the code is committed.
 
-
 When a new bug is found
 -----------------------
 A new test case should be added by imitating the style of an existing
@@ -60,7 +138,6 @@ XFAIL_TESTS lists and new-bug-ref.png to EXTRA_DIST. Add new-bug to
 .gitignore, and last but not least, don't forget to "git add" the new
 files.
 
-
 When a new feature is added
 ---------------------------
 It's important for the regression suite to keep pace with development
@@ -77,7 +154,6 @@ CAIRO_REF_DIR to point at a previous test directory, relative
 to the current test directory, and any previous output will be
 used by preference as reference images.
 
-
 When a bug is fixed
 -------------------
 The fix should be verified by running the test suite which should
@@ -85,7 +161,6 @@ result in an "unexpected pass" for the test of interest. Rejoice as
 appropriate, then remove the relevant file name from the XFAIL_TESTS
 variable in Makefile.am.
 
-
 Before releasing
 ----------------
 All tests should return a result of PASS for all supported (those enabled by
@@ -95,7 +170,7 @@ message:
 	All XX tests passed
 
 Some notes on limitations in poppler
-------------------------------------
+====================================
 One of the difficulties of our current test infrastructure is that we
 rely on external tools to convert cairo's vector output (PDF,
 PostScript, and SVG), into an image that can be used for the image
