[cairo] cairo test suite suggestions

[Daniel Goldman]

Hi,

I compiled the cairo test suite source and ran it with "make test". It's obvious 
that a huge amount of effort has gone into this. It's really good. But I had a 
few suggestions:

- PUBLISH - Are the index.html and output files on the cairo web site? I didn't 
see them. I think it would be quite helpful to have the output on the cairo web 
site. If not on the web site, and reason they are not there?

- ADD PDF - I didn't see any pdf output. I tried running "make test 
TARGETS=pdf". But it kept saying "cannot find target pdf". Any reason not to 
produce pdf output by default? And what might I be doing wrong?

- ADD LEGEND - The output boxes are green, yellow, red, white, and grey. I would 
suggest having a simple legend spelling out exactly what each color means. I 
would say green and red are self-explanatory, but other colors are not really 
self-explanatory. And there is no reason to be coy. Would any disagree?

- DOCUMENT - I understand that testing for correct operation has to be the first 
priority. But this could also be a useful way to show what cairo does and 
illustrating program options. It already does that to some extent, but not that 
much. Related to that, I would include some text with each row to explain the 
purpose of the test. Perhaps (yes, perhaps) it's clear to the current developers 
what each test does, but it's almost totally unclear to anyone else. It seems to 
me the text could be included at the top of the log file.

- LOG SECTIONS - In the logfiles, I would put a blank line after each TEST line, 
to make the file MUCH easier to read. Seems very simple to do.

- ADD SUBDIRS - After running the suite, there are almost 6000 files in the 
directory. I think most would agree this is rather excessive and awkward. I 
would put test suite results into subdirectories based on output type. Does 
anyone like 6000 files in a directory?

- SPLIT LOGS - Many of the log files combine results from different rows. I 
would  split them out, to make it easier to match up the results with the log  
file. This might take more effort, and seems lower priority.

Thanks,
Daniel