[cairo] Does the performance suite work?

[BJörn Lindqvist]

Hello all,

It seems like the cairo-perf program doesn't work at all. The variance
in the results I get are so big that it can't be used to measure
performance adequately.

Either that, or maybe I'm just not interpreting the results correctly.
Check this out (filtered for brevity):

$ CAIRO_TEST_TARGET=image ./perf/cairo-perf -i 100 paint_radial_rgba_over
[ # ]  backend-content                    test-size min(ticks)
min(ms) median(ms) stddev. iterations
[  1]    image-rgb      paint_radial_rgba_over-512   67486740 4924.274
4937.784  1.01%  97

Not sure what this means.. Does the paint_radial_rgba_over takes
4924.274 ms for 97 iterations or for one? What does min(ticks)
measure? Here are the results for some more runs:

[  1]    image-rgb      paint_radial_rgba_over-512   68211688 3411.400
3420.407  0.61%  88
[  1]    image-rgb      paint_radial_rgba_over-512   67554741 5040.423
5054.104  0.51%  83
[  1]    image-rgb      paint_radial_rgba_over-512   67692931 3737.492
3746.974  0.53%  85
[  1]    image-rgb      paint_radial_rgba_over-512   68417333  791.231
 794.300  0.90%  99
[  1]    image-rgb      paint_radial_rgba_over-512   67712492 4822.050
4836.170  0.70%  90

As you can see, this makes totally no sense. The median(ms) value
varies between 794-5054. The variance is this big even when I run with
1000 iterations or more.


-- 
mvh Björn