[cairo] Some preliminary coverage results for the cairo test suite

[Carl Worth]

Today I started playing with gcov and I came up with the following
results for the test suite's coverage of cairo/src:

	http://cairographics.org/~cworth/cairo-coverage-2005-12-14/

Thanks to lcov[1] for the lovely HTML rendering of the results.

I'm quite happy to see that our current statement coverage is as high
as 70%. I really didn't expect anything that high at all given the
ad-hoc way in which we've been writing tests.

Now that we have this coverage report, it should be quite easy to find
untested code, (which is broken by definition), and add tests to the
test suite to exercise that code and start finding some of the lurking
bugs. This would be an excellent activity for anyone that has been
itching to get their hands on the cairo implementation, but didn't
have any good idea of where to start.

I'd like to automate the generation of reports like this, eventually
having something on the web that's always up to date with the code in
CVS. Getting that far will also require getting some automated
build-cairo-and-run-the-test-suite thing setup, (which I've also
wanted).

An earlier step would be to get the Makefiles setup to generate this
report. Here's some of what I did to get this far, if anyone else
would like to attempt to replicate it:

1) gcov itself comes with gcc, so if you're using gcc already you're
   done with step 1. If you're not, then stop reading as the remaining
   steps won't help you.

2) The code to be profiled needs to be compiled with two additional
   options and without optimization:

	-fprofile -ftest-coverage -O0

3) The tests themselves don't need to be compiled with these options
   (the coverage of the tests is not interesting), but any programs
   linked with do need to be linked with -lgcov.

4) I discovered that using ccache breaks these gcc options, (or
   strictly speaking, they still work but their output ends up in the
   ccache cache directories rather than the current directory). I
   turned off ccache with:

	make CC=/usr/bin/gcc

5) I had trouble getting a shared library to compile this way, so I
   configured with --disabled-shared to get the results.

6) After running "make check" I ran the following commands in
   cairo/src:

	gcov *.c
	lcov --directory . -z
	lcov --directory . -c -o cairo.lcov
	genhtml -o cairo-coverage-`date +%F` cairo.lcov

   The genhtml program comes with lcov, (it comes with a variety of
   utility program, each equally poor in non-namespace naming:
   geninfo, genpng, gendesc).

If anyone would like to contribute some appropriate Makefile magic to
automate more of this, that would be great.

-Carl

[1] http://ltp.sourceforge.net/coverage/lcov.php

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.freedesktop.org/archives/cairo/attachments/20051214/07976c26/attachment.pgp