[cairo] ANN: "Caskbench" performance benchmark

[Chris Wilson]

On Fri, Sep 26, 2014 at 01:01:25PM -0700, Bryce Harrington wrote:
> On Fri, Sep 26, 2014 at 07:33:03AM +0100, Chris Wilson wrote:
> > On Thu, Sep 25, 2014 at 05:54:40PM -0700, Bryce Harrington wrote:
> > > At Samsung one of the projects I've been working on is a benchmark test
> > > for comparing the performance of Cairo and Skia with EGL + MSAA, called
> > > Caskbench.  I presented about this testing at LinuxConf US in Chicago
> > > last month.
> > > 
> > > This is no where near as comprehensive or meticulous as Cairo's
> > > performance test suite, but it runs quickly, and includes Skia ports of
> > > each of the tests.  The idea here being to do fair apples-to-apples
> > > comparisons of the two codebases, or for comparing performance of
> > > Cairo's image backend with the egl backend.
> > 
> > In the stock configuration you don't have a fair comparison as they have
> > different rendering intent and substantially different levels of
> > quality. At the same resolution, cairo's scanline converter is
> > marginally faster, but skia benefits from a better spline walker and
> > inline rasterisation. Not to mention that skia has a hairline stroker
> > and imprecise gradient caches but has a hard time rasterising rectilinear
> > polygons (or at least did a couple of years ago).
> 
> Right, there's a LOT of assumptional differences between the two.  Even
> simple stuff like whether to alpha blend by default and pixel
> alignment.  I try to correct for some of this, when there's
> user-accessible settings.  But where one renderer has better internals I
> figure they deserve credit for that... and I'd wonder if that's
> something we could look at doing in Cairo?

Sure, it was the whole reasoning behind pushing for fast modes in
geometry rasterisation et al. By default, they are not used but are
available if the user want to trade quality for speed.

Of course, we still need the improvements for spline handling,
specialised strokers, offset curves, arcs, integration between
scanline converter and rasteriser, etc. 

Also my focus has been on applications rather than microbenchmarks. The
cairo api and our state tracking is fairly heavyweight, which tends to
show up as excessive in synthetics, but does help in the traces we
capture.

Any improvement is great of course.
  
> > > The codebase has been open sourced and is available on github:
> > > 
> > >   https://github.com/Samsung/caskbench
> > 
> > How do you control the PRNG? And as you are using rand() that often, you
> > probably want to consider an alternate LCG rng.
> 
> Yes, this is high on the todo list.  In fact I've found some hardware
> where the prng can't be re-seeded, so I need to replace all that
> anyway.
> 
> 
> Anyway, ignoring skia and looking just at Cairo, here's what I'm getting
> for comparing Cairo's image backend to its EGL backend.  This is on a
> Sandybridge box with ubuntu 14.04 (which uses mesa 3.0):
> 
>                          image   egl
> --------------------------------------------
> cairo-animation            288   290      0%
> cairo-bubbles               83   495    496%
> cairo-circle               112   274    144%
> cairo-clip                 556  2953    431%
> cairo-cubic                151   280     85%
> cairo-curves                18    29     61%
> cairo-fill                 182   435    139%
> cairo-hline               1055  1156      9%
> cairo-image                256  3870   1411%
> cairo-image_rotate          69  2947   4171%
> cairo-image_scale          140  3214   2195%
> cairo-linear_gradient       22  1184   5281%
> cairo-line                 497   667     34%
> cairo-mask                 442   888    100%
> cairo-multi_line           369   927    151%
> cairo-multi_shape          178   349     96%
> cairo-paint                 62  1357   2088%
> cairo-quadratic            188   313     66%
> cairo-radial_gradient       11  1167  10509%
> cairo-rect                  50   629   1158%
> cairo-roundrect            187   261     39%
> cairo-shadow                 4     2    -50%  # (Feature unimplemented)
> cairo-star                 565   223    -60%
> cairo-text                 330   841    154%
> cairo-text_glyphs          510  1116    118%
> cairo-transform             16    22     37%
> cairo-vline               1086  1172      7%
> 
> The numbers are fps so bigger is better.
> 
> Apart from the stars test, cairo-gl did better all-around than cairo
> image.  And for gradients and image manipulation it's profoundly faster.
> Now, I haven't tested against GLX but this makes me wonder if this would
> translate into significantly better performance for, say, Inkscape?  If
> so, should we maybe be advocating cairo-gl/cairo-egl more heavily here?

I would say that they are, in a large part at least, suspicously good.
Time to check on how mesa has progressed.
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre