62

I have just started writing some unit tests for a python project I have using unittest and coverage. I'm only currently testing a small proportion, but I am trying to work out the code coverage

I run my tests and get the coverage using the following

python -m unittest discover -s tests/
coverage run -m unittest discover -s tests/
coverage report -m

The problem I'm having is that coverage is telling I have 44% code coverage and is only counting the files that:

  1. were tested in the unit tests (i.e., all the files that were not tested are missing and not in the overall coverage)

  2. were in the libraries in the virtual environment and code coverage of the actual tests too. Surely it should not be including the actual tests in the results?

Furthermore, it says the files that are actually tested in these unit tests only have the first few lines tested (which are in most cases the import statements)

How do I get a more realistic code coverage or is this how it is meant to be?

nbro
  • 15,395
  • 32
  • 113
  • 196

4 Answers4

72

Add --source=. to the coverage run line. It will both limit the focus to the current directory, and will search for .py files that weren't run at all.

nbro
  • 15,395
  • 32
  • 113
  • 196
Ned Batchelder
  • 364,293
  • 75
  • 561
  • 662
  • 12
    Thank you, this works for me! `coverage run --source={dir} -m unittest discover -s tests/` will cover {dir}. Can I ask though: What about going through the subdirs (recursively)? – sunyata Sep 19 '17 at 15:50
  • I found that it's possible to use this to include multiple directories/modules: `--source={dir1},{dir2},{etc}` (please note that there's no space after the comma signs). It still would be useful with a recursive option if that is available though in case new modules are added to the source code – sunyata Sep 19 '17 at 16:59
  • 1
    Documentation: [Coverage.py command line usage](https://coverage.readthedocs.io/en/latest/cmd.html) – sunyata Sep 19 '17 at 17:16
  • 1
    `--source={dir}` will find the importable modules anywhere in the tree rooted at dir. – Ned Batchelder Sep 19 '17 at 19:20
  • 2
    Okay, but this doesn't work for me, i have version 4.4.1 and am running on Ubuntu 16.04 with this command: `coverage run --source=./mc -m unittest discover -s tests` (where `mc` is the root package directory). I have to add all the subdirs to the root {dir} for it to work: `coverage run --source=./mc,./mc/win -m unittest discover -s {test-dir}` – sunyata Sep 19 '17 at 20:25
  • Alright, i have now figured out the problem, i hadn't added a `__init__.py` file in the `/win` subdirectory, after doing this everything within the directory is included so i only have to specify `./mc` after `--source` – sunyata Sep 19 '17 at 23:47
  • this is still quite unclear. Can anyone who has this working specify what is needed to generate the right coverage report? I think it goes without saying that we need every python file in the working directory included, whether it has any unit tests or has an __init__.py or not. – TamaMcGlinn Feb 12 '21 at 15:28
  • 1
    Currently, coverage.py will only report on what it considers to be importable. It looks for `__init__.py` files in subdirectories to know if they are importable. This is so "bin" directories don't get flagged as uncovered. Perhaps we need an option to not care about `__init__.py` files. – Ned Batchelder Feb 12 '21 at 19:13
  • This is great! But, it makes my coverage number go waaaaay down. Must be a bug in `coverage`. >_ – cod3monk3y Apr 01 '23 at 02:07
2

If you use nose as a testrunner instead, the coverage plugin for it provides

  --cover-inclusive     Include all python files under working directory in
                        coverage report.  Useful for discovering holes in test
                        coverage if not all files are imported by the test
                        suite. [NOSE_COVER_INCLUSIVE]

  --cover-tests         Include test modules in coverage report
                        [NOSE_COVER_TESTS]
Daenyth
  • 35,856
  • 13
  • 85
  • 124
1

Adding --source=. won't work if you don't have explicit __init__.py files in every directory containing .py files.

As Ned Batchelder pointed out in a comment to his answer, coverage.py looks for these __init__.py files to know if the contents of directory are importable. (code src on github)

Follow up reading:
github pytest-cov issue
github coveragepy issue
solution of enforcing __init__.py files

Igor Kołakowski
  • 109
  • 6
0

You can find this in coverage docs

add the following to coveragerc

[report]
include_namespace_packages = True

(This might include some unwanted packages as well, use omit config to remove those)

Vaibhav
  • 3
  • 3