Coverage.py

I finally had a chance to try coverage.py, a Python code coverage tool which is maintained by Ned Batchelder. I’m liking it a lot: it’s minimal fuss to set up and use, yet very useful indeed.

When you’ve installed coverage, it should be on your path. Taking my work-in-progress PyGame code as an example, I ran coverage on the test suite:

$ coverage run run_tests.py

The test suite runs as normal:

$ coverage run run_tests.py
.................................................
----------------------------------------------------------------------
Ran 49 tests in 0.006s

OK
$

Coverage writes the results to a .coverage file. To view this as a plain text report:

$ coverage report
Name                      Stmts   Exec  Cover
---------------------------------------------
constants                     6      6   100%
domain/__init__               1      1   100%
domain/deployment            27     23    85%
domain/game_date             48     48   100%
domain/unit                  18     14    77%
infrastructure/__init__       1      1   100%
infrastructure/storage       12     12   100%
run_tests                     4      4   100%
tests/__init__                3      3   100%
tests/test_deployment        94     94   100%
tests/test_game_date         64     64   100%
tests/test_unit              57     57   100%
---------------------------------------------
TOTAL                       335    327    97%
$

But for the best results, you can create an HTML report:

$ coverage html
$

This writes HTML to an htmlcov directory, which can be viewed in a browser. The HTML report provides not only the summary listed in the plain text report, but links to pages for each source file. Each source listing is highlighted to indicate where code coverage is present and where code coverage is missing. Simple, but very, very useful. I’ve used the reports to improve the test suite’s coverage, and even identified code that could be removed. As you can see above, domain/deployment.py and domain/unit.py need better code coverage, which is something I’m aware needs to be addressed next as it will help with some upcoming changes.

One caveat is that coverage can only provide code analysis for code touched by your test suite. There are two areas of the code missing from the above report: the first is the source that binds the game together, since it’s not referenced by the test suite. More importantly, I mentioned I don’t have UI tests yet: since no UI code is called by the test suite, it doesn’t show on the report.

Something else to mention is it only tells you tests have touched sections of code – it’s up to you to ensure your test suite properly exercises that code.

Very cool, and thought I’d share in case you hadn’t seen this tool before.

Coverage can be obtained from http://pypi.python.org/pypi/coverage

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s