[cairo] A better image comparison tool

[Behdad Esfahbod]

Ralph Giles wrote:
> On Mon, Sep 22, 2008 at 11:53 AM, Antoine Azar <cairo at antoineazar.com> wrote:
> 
> We'd be interested to hear how this goes. The problem we've had with a
> lot of these measures is that the were developed for image recognition
> and video compression applications and don't apply well to
> high-contrast vector graphics

Exactly.  I don't think any stock approaches are going to help here.  And
pdiff is living evidence of that.

> We use a threshold+cluster tool to cull single-pixel differences in
> our regression tests for ghostscript. It counts the number of pixels
> whose values differ by more than a threshold value, and then counts
> how many of those are in clusters larger than a particular window
> size.

Interesting.


I think for many of the tests what we are more interested in is "is it there",
rather than "does every pixel match".  For those kind of tests I guess the
following should work:

  - For each channel of each pixel in each of the images check that its value
lies in between of the values of its neighborhood in the other image.

Where neighborhood can be defined as 1-pixel for exact match, 5-pixel or
9-pixel for fuzzy matches.

This should be enough to address all the aa rendering artifacts we've been
facing.  There are issues with the SVG backend where it converts double colors
to bytes differently and hence has one-off differences for some colors.  We
currently have a tolerance per target for that.  With ickle's reworked test
suite we can adjust all the knobs on a per-test per-target basis which should
help minimize false-negatives.


behdad