[cairo] ANN: "Caskbench" performance benchmark

[Bryce Harrington]

On Fri, Sep 26, 2014 at 07:33:03AM +0100, Chris Wilson wrote:
> On Thu, Sep 25, 2014 at 05:54:40PM -0700, Bryce Harrington wrote:
> > At Samsung one of the projects I've been working on is a benchmark test
> > for comparing the performance of Cairo and Skia with EGL + MSAA, called
> > Caskbench.  I presented about this testing at LinuxConf US in Chicago
> > last month.
> > 
> > This is no where near as comprehensive or meticulous as Cairo's
> > performance test suite, but it runs quickly, and includes Skia ports of
> > each of the tests.  The idea here being to do fair apples-to-apples
> > comparisons of the two codebases, or for comparing performance of
> > Cairo's image backend with the egl backend.
> 
> In the stock configuration you don't have a fair comparison as they have
> different rendering intent and substantially different levels of
> quality. At the same resolution, cairo's scanline converter is
> marginally faster, but skia benefits from a better spline walker and
> inline rasterisation. Not to mention that skia has a hairline stroker
> and imprecise gradient caches but has a hard time rasterising rectilinear
> polygons (or at least did a couple of years ago).

Right, there's a LOT of assumptional differences between the two.  Even
simple stuff like whether to alpha blend by default and pixel
alignment.  I try to correct for some of this, when there's
user-accessible settings.  But where one renderer has better internals I
figure they deserve credit for that... and I'd wonder if that's
something we could look at doing in Cairo?
 
> > The codebase has been open sourced and is available on github:
> > 
> >   https://github.com/Samsung/caskbench
> 
> How do you control the PRNG? And as you are using rand() that often, you
> probably want to consider an alternate LCG rng.

Yes, this is high on the todo list.  In fact I've found some hardware
where the prng can't be re-seeded, so I need to replace all that
anyway.


Anyway, ignoring skia and looking just at Cairo, here's what I'm getting
for comparing Cairo's image backend to its EGL backend.  This is on a
Sandybridge box with ubuntu 14.04 (which uses mesa 3.0):

                         image   egl
--------------------------------------------
cairo-animation            288   290      0%
cairo-bubbles               83   495    496%
cairo-circle               112   274    144%
cairo-clip                 556  2953    431%
cairo-cubic                151   280     85%
cairo-curves                18    29     61%
cairo-fill                 182   435    139%
cairo-hline               1055  1156      9%
cairo-image                256  3870   1411%
cairo-image_rotate          69  2947   4171%
cairo-image_scale          140  3214   2195%
cairo-linear_gradient       22  1184   5281%
cairo-line                 497   667     34%
cairo-mask                 442   888    100%
cairo-multi_line           369   927    151%
cairo-multi_shape          178   349     96%
cairo-paint                 62  1357   2088%
cairo-quadratic            188   313     66%
cairo-radial_gradient       11  1167  10509%
cairo-rect                  50   629   1158%
cairo-roundrect            187   261     39%
cairo-shadow                 4     2    -50%  # (Feature unimplemented)
cairo-star                 565   223    -60%
cairo-text                 330   841    154%
cairo-text_glyphs          510  1116    118%
cairo-transform             16    22     37%
cairo-vline               1086  1172      7%

The numbers are fps so bigger is better.

Apart from the stars test, cairo-gl did better all-around than cairo
image.  And for gradients and image manipulation it's profoundly faster.
Now, I haven't tested against GLX but this makes me wonder if this would
translate into significantly better performance for, say, Inkscape?  If
so, should we maybe be advocating cairo-gl/cairo-egl more heavily here?

Bryce