[cairo] What does it take to get a make check to pass with the xcb target, using CAIRO_REF_DIR?

[darxus at chaosreigns.com]

On 06/30, Bryce Harrington wrote:
> We do know there are certain tests with race conditions that will
> alternate between passing and failing in two identical, sequential test
> runs.  Those test cases you mention don't ring a bell, but if you see
> it's the same set doing that each time, then that could be the case.  I
> don't know what to do about those, and would not oppose disabling those
> test cases (although I assume they were meant to test something
> important...)

Yes, these failures are definitely indication of a bug.  The few I looked
at looked like glitches in transparency and anti-aliasing.  So opening a
bug pointing out what was disabled seems appropriate.

What would be the best way to disable these tests?  I tried making them
xfail, by copying the corresponding test/output/*out.png files to
test/reference/*xfail.png but I'm still getting FAILs, I suspect because
the output has to exactly match the xfail.png, which I do not expect it to.

Delete them from test/Makefile.sources ?

Removing these test from test/Makefile.sources allows make check to pass
(with an updated set of references):  record1414x.c record2x.c record90.c
recordflip.c record.c text-rotate.c

Would a patch for this be accepted?  Should it be different than the one
below?


Subject: [PATCH] Remove tests which fail differently each time.

---
 test/Makefile.sources | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/test/Makefile.sources b/test/Makefile.sources
index 479b2ca..ac8ecd3 100644
--- a/test/Makefile.sources
+++ b/test/Makefile.sources
@@ -266,11 +266,6 @@ test_sources = \
 	random-intersections-curves-eo.c		\
 	random-intersections-curves-nz.c		\
 	raster-source.c					\
-	record.c					\
-	record1414x.c					\
-	record2x.c					\
-	record90.c					\
-	recordflip.c					\
 	record-extend.c					\
 	record-neg-extents.c                            \
 	record-mesh.c					\
@@ -352,7 +347,6 @@ test_sources = \
 	text-cache-crash.c				\
 	text-glyph-range.c				\
 	text-pattern.c					\
-	text-rotate.c					\
 	text-transform.c				\
 	text-unhinted-metrics.c				\
 	text-zero-len.c					\
@@ -388,6 +382,14 @@ test_sources = \
 	zero-alpha.c					\
 	zero-mask.c
 
+# Disabled because they fail differently each time:
+#	record1414x.c					\
+	record2x.c					\
+	record90.c					\
+	recordflip.c					\
+	record.c					\
+	text-rotate.c					
+
 pthread_test_sources =					\
 	pthread-same-source.c				\
 	pthread-show-text.c				\
-- 
2.7.4