Code coverage does not cover dynamically imported packages/modules

Question

I am using the following command to run tests:

nosetests --with-coverage --cover-html --cover-package mypackage

I would like the coverage report to be updated, even if a developer adds new, untested, code to the package.

For example, imagine a developer adds a new module to the package but forgets to write tests for it. Since the tests may not import the new module, the code coverage may not reflect the uncovered code. Obviously this is something which could be prevented at the code review stage but it would be great to catch it even earlier.

My solution was to write a simple test which dynamically imports all modules under the top-level package. I used the following code snippet to do this:

import os
import pkgutil

for loader, name, is_pkg in pkgutil.walk_packages([pkg_dirname]):
    mod = loader.find_module(name).load_module(name)

Dynamically importing sub-packages and sub-modules like this does not get picked up by the code coverage plugin in nose.

Can anyone suggest a better way to achieve this type of thing?

Answer 1

The problem seems to be the method for dynamically importing all packages/modules under the top-level package.

Using the method defined here seems to work. The key difference being the use of importlib instead of pkgutil. However, importlib was introduced in python 2.7 and 3.1 so this solution is not appropriate for older versions of python.

I have updated the original code snippet to use __import__ instead of the ImpLoader.load_module method. This also seems to do the trick.

import os
import pkgutil

for loader, name, is_pkg in pkgutil.walk_packages([pkg_dirname]):
    mod = loader.find_module(name)
    __import__(mod.fullname)