[cairo] Benchmarking Cairo

[M Joonas Pihlaja]

On Sat, 16 Aug 2008, Nicholas Allen wrote:

> I couldn't find much benchmarking of Cairo (at least not for recent
> versions). Is this not done before each release? If not, why not?

There's a performance suite (separate from the correctness test
suite) in the code base that is used to check for speed
regressions.  It's well integrated into the version control
system, so you can compare the effects of individual commits if
you like, or compare releases and backends against each other if
you want.

> I don't know too much about the development process of Cairo but I
> assume it is possible to commit to trunk with a failing test or
> performance problems.

Sure, mistakes happen.

In some sense the test suite is a victim of its own success:
there are now so many tests that runnning the test suite is
painful, so presumably it doesn't get run as often as it should.
Compared to a build bot / gatekeeper, I think working on user
friendlyness of the test suite (e.g. make it parallelisable)
would have a larger immediate impact.

I'd guess that an onerous commit process would make for bigger or
less frequent commits, though, and I can't see either of those
being a good thing.

Cheers,

Joonas