<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"></head><body>Those tests were not useful, because I ran them against references I generated from the latest commit. It'll be several days before I can try something more useful. -------- Original message -------- From: darxus@chaosreigns.com Date: 07/08/2016 8:58 AM (GMT-05:00) To: Bryce Harrington <bryce@osg.samsung.com> Cc: cairo@cairographics.org Subject: Re: [cairo] What does it take to get a make check to pass with the xcb target, using CAIRO_REF_DIR? On 07/07, Bryce Harrington wrote: > When doing releases I also try to do a run and compare with the prior > release to spot regressions. I'd really love to be able to do a > thorough git bisect search to locate which changes caused which tests to > fail, but as the suite takes a while to run it'd be pretty time > consuming. I think bisecting sounds like great fun, except for the irregularity of problems. I wrote a script to build a random commit, back through 2010 (because they seemed to not be successfully building before that). All of them either gave me no test-suite.log, or, for the 22 that gave me a log, the test suite had at least 6 failures. Nothing passed all tests. Does anybody have any ideas when all tests for the xcb target might have passed? Is this evidence that the FAILs are caused by an external library? This is the output, plus the perl script I wrote: http://www.chaosreigns.com/cairo/tests.tar.gz (The script is kind of crude.) This is a filename with a unix timestamp (git log -1 --pretty=format:%ct), an underscore, the commit hash, a colon, the number of FAILs, some spaces, then a GMT date calculated from the unix timestamp: darxus@dancer:~/source/cairolog$ grep -l test * | xargs grep -c ' FAIL' 1264201313_c0458b456007f718747be7fd690e674df5026059:7 Fri Jan 22 23:01:53 2010 1269204078_247ef2bd63e755963f7930ac06c79b95aed2adb4:6 Sun Mar 21 20:41:18 2010 1271277994_54670ec13d64efa94f552b5473c1f15a9db1cecd:145 Wed Apr 14 20:46:34 2010 1273693338_094f0e0fa0153f290061635eed51e8d1dbe2cf4a:5 Wed May 12 19:42:18 2010 1276288178_d31e2418d6d9920205a154f21becca12a7b84c2a:7 Fri Jun 11 20:29:38 2010 1278883932_fbb0a260b707cb5f02a14cc368c6f2f0d63564c3:146 Sun Jul 11 21:32:12 2010 1286969384_90d50cd92315d6760069ad8062aba5e297370b20:6 Wed Oct 13 11:29:44 2010 1292229969_d9ba8337ab456ae0e232d3c603cb41cea984ebea:32 Mon Dec 13 08:46:09 2010 1292427140_e4b78424ac82588bcb9b855d5b6d5872050d33f9:7 Wed Dec 15 15:32:20 2010 1297614844_c3605bd3adcae1f12731230a1ea599d15e3c8cad:7 Sun Feb 13 16:34:04 2011 1311937629_58df191946b7d275c5706dc40e1cf3dffa7cceae:8 Fri Jul 29 11:07:09 2011 1313177159_bb24f165ff57973347b34956a371c6b33d2d9b59:55 Fri Aug 12 19:25:59 2011 1342113299_3ed581149f82b653e1987e746467ad8ae9ce3ca2:7 Thu Jul 12 17:14:59 2012 1348565464_31291611625c21bc9ec2827cf4ccc8a1f18392ed:55 Tue Sep 25 09:31:04 2012 1405128883_d5ffe67008afe46c030b31afc0803321b6479990:7 Sat Jul 12 01:34:43 2014 1413893554_f032133e6d5ad05157fc46609d8c63103028342c:7 Tue Oct 21 12:12:34 2014 1423145626_3cf862f6d973755cd9824c2224fbe0a623c47ff1:7 Thu Feb 5 14:13:46 2015 1425713393_58df191946b7d275c5706dc40e1cf3dffa7cceae:7 Sat Mar 7 07:29:53 2015 1434656331_ae608035c7b7133826a608d45e067c3875a1aceb:7 Thu Jun 18 19:38:51 2015 1435360353_70cc8f250b5669e757b4f044571ba0f71e3dea9e:7 Fri Jun 26 23:12:33 2015 1465125216_caa4c9fdeb3aecd9a4288114e75d24ec931cd01b:6 Sun Jun 5 11:13:36 2016 1467845160_f9b65ae1fc91bc558a01c2ad7be5a121c6f10818:7 Wed Jul 6 22:46:00 2016 Of the ones that had fewer than 10 FAILs, the test names mostly started with "record", with a few "subsurface". The most recent one with no test-suite.log was 1466331752_bf5adaf3942388e58ad3bda30173e53b214df885 Sun Jun 19 10:22:32 2016 Which I manually checked, and it did fail to build: test/cairo-test-runner.c:130: undefined reference to `cairo_boilerplate_xmalloc' The second most recent one with no test-suite.log was: 1465125216_afe6f4f0519606c4bc7e9b705b0cae75692d7af2 Sun Jun 5 11:13:36 2016 It also failed to build: ./autogen.sh: running `./configure --cache-file=config.cache --disable-static --enable-test-surfaces --prefix=/home/darxus/install --enable-xcb' configure: creating cache config.cache configure: error: cannot find install-sh, install.sh, or shtool in "." "./.." "./../.." (Which, so far, confirms my expectation that empty output files are due to failures to build.) This is how many I attempted to build per year that produced no test-suite.log. The ones from before 2010 were from before I started to restrict the date range (and go to sleep): $ grep -L test * | cut -d'_' -f1 | ~/bin/unixtime.pl | awk '{print $5}' | uniq -c 1 2002 3 2003 3 2004 16 2005 14 2006 15 2007 23 2008 6 2009 100 2010 80 2011 40 2012 22 2013 18 2014 7 2015 2 2016 This is how many total I ran: darxus@dancer:~/source/cairolog$ ls *_* | cut -d'_' -f1 | ~/bin/unixtime.pl | awk '{print $5}' | uniq -c 1 2002 3 2003 3 2004 17 2005 14 2006 15 2007 23 2008 6 2009 109 2010 84 2011 42 2012 22 2013 20 2014 11 2015 4 2016 </body></html>