[cairo] cairo test suite suggestions
dagoldman at yahoo.com
Sat Aug 21 15:16:55 PDT 2010
I compiled the cairo test suite source and ran it with "make test". It's obvious
that a huge amount of effort has gone into this. It's really good. But I had a
- PUBLISH - Are the index.html and output files on the cairo web site? I didn't
see them. I think it would be quite helpful to have the output on the cairo web
site. If not on the web site, and reason they are not there?
- ADD PDF - I didn't see any pdf output. I tried running "make test
TARGETS=pdf". But it kept saying "cannot find target pdf". Any reason not to
produce pdf output by default? And what might I be doing wrong?
- ADD LEGEND - The output boxes are green, yellow, red, white, and grey. I would
suggest having a simple legend spelling out exactly what each color means. I
would say green and red are self-explanatory, but other colors are not really
self-explanatory. And there is no reason to be coy. Would any disagree?
- DOCUMENT - I understand that testing for correct operation has to be the first
priority. But this could also be a useful way to show what cairo does and
illustrating program options. It already does that to some extent, but not that
much. Related to that, I would include some text with each row to explain the
purpose of the test. Perhaps (yes, perhaps) it's clear to the current developers
what each test does, but it's almost totally unclear to anyone else. It seems to
me the text could be included at the top of the log file.
- LOG SECTIONS - In the logfiles, I would put a blank line after each TEST line,
to make the file MUCH easier to read. Seems very simple to do.
- ADD SUBDIRS - After running the suite, there are almost 6000 files in the
directory. I think most would agree this is rather excessive and awkward. I
would put test suite results into subdirectories based on output type. Does
anyone like 6000 files in a directory?
- SPLIT LOGS - Many of the log files combine results from different rows. I
would split them out, to make it easier to match up the results with the log
file. This might take more effort, and seems lower priority.
More information about the cairo