[cairo] [PATCH 0/3] test: Add test for image downscaling

[Bryce W. Harrington]

Hi Chris, for the image downscaling patch Krzysztof Kosiński posted
a few weeks ago, you mentioned we'd need some corresponding
tests.  I've taken a shot at that; this patchset includes some simple
tests for three scaling factors, for each of the filter options.

I plan to also do downscaling with skew and repeat, but wanted to post
what I have for your review before I get too far along.  In particular,
I'm curious what I should do for the reference images; I generated them
using Gimp as 'truth' but the anti-aliasing results are different, and
I'm not sure what should be taken as correct behavior.

This patchset is also posted to the 'downscaling' branch in my fdo git
repo:

  http://cgit.freedesktop.org/~bryce/cairo/


Test Results
------------

Here are the current results of testing with this patchset:

 Passed:           267	     267
 Failed:           153	     153
 Expected Failed:  0  	     0  
 Error:            0  	     0  
 Crashed:          0  	     0  
 Untested:         0  	     0  
 Total:            420	     420

The first column is cairo built with pixman 0.24.4, the second is with
pixman 0.31.1; both builds produced identical results.

60 of the failures are the ps2/ps3 tests.  They're failing across the
board; it looks like it's blurring the edges even when it's not
necessary.

10 tests are the image16/rgb24 test case.  I'm guessing the 16 means it
has a 16 color palette and some of the colors get flattened.  So maybe
this is just a limitation of the format and the reference image should
be adjusted.  But I'm not sure why the 1:4 scaled cases are passing, if
that's the case?

An additional 14 test failures occur for the svg, pdf, and script
backends with the fast and nearest filters (same thing, really) when
downscaling from 96 to 95 pixels.  The difference here is very modest;
the pdiff shows pixel differences but visually they look identical.

The bilinear/95, best/95, good/95 test cases represent the remaining 69
fails.  These are sort of intentional fails, since this is what
Krzysztof Kosiński's downscaling-with-convolution patch is addressing.
Also, the Interpolation options I used to produce the reference images
are probably not correct.



Testing of Proposed Patch
-------------------------

I also did a preview comparison test run using Krzysztof Kosiński's
proposed downscaling patch to use Pixman's convolution filters:

 Passed:           174
 Failed:           246
 Expected Failed:  0
 Error:            0
 Crashed:          0
 Untested:         0
 Total:            420

Only runs against cairo with pixman 0.31.1 were done, since the
convolution downscaling feature isn't present in pixman 0.24.

This doesn't look so good...  All the failures I mentioned above are
still there, plus another 93 new failures.

Of these, 75 are the 1:4 scaling tests.  Whereas previously these were
being cleanly scaled with no border blur, now the edges have a slight
blurring.  This should be investigated more before the patch is
accepted.

The other 18 new failures are with the nearest/95, fast/95 filters.
Here, the shading color selected for the edge blurs differ for some of
the backends.  I'm not sure if this represents a regression or not, but
should get additional analysis to be sure.



Bryce Harrington (3):
  test: Add test for image downscaling
  test: Test a variety of scales when downscaling
  test: Exercise image scaling quality when downscaling

 test/Makefile.sources                              |    1 +
 test/pixman-downscale.c                            |  199 ++++++++++++++++++++
 test/quad-color.png                                |  Bin 0 -> 301 bytes
 test/reference/pixman-downscale-best-24.ref.png    |  Bin 0 -> 191 bytes
 test/reference/pixman-downscale-best-95.ref.png    |  Bin 0 -> 1580 bytes
 test/reference/pixman-downscale-best-96.ref.png    |  Bin 0 -> 301 bytes
 .../reference/pixman-downscale-bilinear-24.ref.png |  Bin 0 -> 191 bytes
 .../reference/pixman-downscale-bilinear-95.ref.png |  Bin 0 -> 640 bytes
 .../reference/pixman-downscale-bilinear-96.ref.png |  Bin 0 -> 301 bytes
 test/reference/pixman-downscale-fast-24.ref.png    |  Bin 0 -> 191 bytes
 test/reference/pixman-downscale-fast-95.ref.png    |  Bin 0 -> 453 bytes
 test/reference/pixman-downscale-fast-96.ref.png    |  Bin 0 -> 301 bytes
 test/reference/pixman-downscale-good-24.ref.png    |  Bin 0 -> 191 bytes
 test/reference/pixman-downscale-good-95.ref.png    |  Bin 0 -> 640 bytes
 test/reference/pixman-downscale-good-96.ref.png    |  Bin 0 -> 301 bytes
 test/reference/pixman-downscale-nearest-24.ref.png |  Bin 0 -> 191 bytes
 test/reference/pixman-downscale-nearest-95.ref.png |  Bin 0 -> 453 bytes
 test/reference/pixman-downscale-nearest-96.ref.png |  Bin 0 -> 301 bytes
 18 files changed, 200 insertions(+)
 create mode 100644 test/pixman-downscale.c
 create mode 100644 test/quad-color.png
 create mode 100644 test/reference/pixman-downscale-best-24.ref.png
 create mode 100644 test/reference/pixman-downscale-best-95.ref.png
 create mode 100644 test/reference/pixman-downscale-best-96.ref.png
 create mode 100644 test/reference/pixman-downscale-bilinear-24.ref.png
 create mode 100644 test/reference/pixman-downscale-bilinear-95.ref.png
 create mode 100644 test/reference/pixman-downscale-bilinear-96.ref.png
 create mode 100644 test/reference/pixman-downscale-fast-24.ref.png
 create mode 100644 test/reference/pixman-downscale-fast-95.ref.png
 create mode 100644 test/reference/pixman-downscale-fast-96.ref.png
 create mode 100644 test/reference/pixman-downscale-good-24.ref.png
 create mode 100644 test/reference/pixman-downscale-good-95.ref.png
 create mode 100644 test/reference/pixman-downscale-good-96.ref.png
 create mode 100644 test/reference/pixman-downscale-nearest-24.ref.png
 create mode 100644 test/reference/pixman-downscale-nearest-95.ref.png
 create mode 100644 test/reference/pixman-downscale-nearest-96.ref.png

-- 
1.7.9.5



--
Bryce Harrington
Senior Open Source Developer  -  b.harrington at samsung.com
Open Source Group             -  Samsung Research America