[cairo] Test suite changes month-by-month this year
Bryce W. Harrington
b.harrington at samsung.com
Thu Apr 10 15:12:35 PDT 2014
I mentioned I was going to check test results weekly. There were no
commits this past week, so instead I did a look back at past test
performance, about once a month since the 1.12.16 release:
1.12 11/26 12/16 1/29 2/27 3/29
Tests run: 24005 24838 24838 24838 24838 24838
Passed: 15959 16167 16164 16166 15883 16725
Failed: 4554 5103 5101 5099 5385 5529
Expected Failed: 336 330 330 330 330 332
Error: 2 0 1 1 1 1
Crashed: 982 1015 1019 1019 1016 28
Untested: 2172 2223 2223 2223 2223 2223
Total: 24005 24838 24838 24838 24838 24838
Listings of what test results changed between each snapshot:
http://www.bryceharrington.org/Cairo/changes-1.12-to-20131126.txt
http://www.bryceharrington.org/Cairo/changes-20131126-to-20131216.txt
http://www.bryceharrington.org/Cairo/changes-20131216-to-20140129.txt
http://www.bryceharrington.org/Cairo/changes-20140129-to-20140227.txt
http://www.bryceharrington.org/Cairo/changes-20140227-to-20140329.txt
Between 1.12.16 and 11/26, a number of surface and fill tests started
failing. I could probably narrow this down further, as there were a
number of significant changes, such as to rounding behavior and
downscaling. But here's the patches that directly changed surface:
$ git log --since "2013-08-21" --until "2013-11-26" --date=short --pretty=format:"%h %cd - %an : %s" | grep ' surface:'
935c64a 2013-09-05 - Alexander Larsson : surface: Inherit device scale in cairo_surface_create_similar()
bc792a5 2013-09-05 - Alexander Larsson : surface: Merge scratch construction into _cairo_surface_create_scratch
bdccf4f 2013-09-05 - Alexander Larsson : surface: Opencode create_similar
7ab34f3 2013-09-05 - Alexander Larsson : surface: expose the device scale
The result changes between 11/26 and 12/16 look to be just test cases
that randomly crash. I believe these are all fixed up now.
Ditto for the 12/16-1/29 results.
1/29-2/27 just are from the mesh regression I introduced in 44a09f462c
and that was fixed in bb174036.
The 2/27-3/29 results are Uli's fix to the mesh bug, and fixes to a
bunch of crashes.
Bryce
More information about the cairo
mailing list