9

In django 1.5 and earlier, running python manage.py test would, by default, run all tests in a project (including all those in django.contrib). Subsequent to version 1.6, the default behaviour is to run all the tests in the current directory.

What is the best way (v 1.6) to run all tests, either with or without the django.contrib tests?

Luke
  • 404
  • 5
  • 11

2 Answers2

15

Django 1.6 changed the default test runner to:

TEST_RUNNER = 'django.test.runner.DiscoverRunner'

You can get the old behaviour back by adding to your settings.py:

TEST_RUNNER = 'django.test.simple.DjangoTestSuiteRunner'

As explained in the release notes:

The previous runner (django.test.simple.DjangoTestSuiteRunner) found tests only in the models.py and tests.py modules of a Python package in INSTALLED_APPS.

The new runner (django.test.runner.DiscoverRunner) uses the test discovery features built into unittest2 (the version of unittest in the Python 2.7+ standard library, and bundled with Django). With test discovery, tests can be located in any module whose name matches the pattern test*.py.

The new runner expects a list of dotted path of modules where tests shall be discovered, so you can also run the tests from django contrib this way:

python manage.py test myproject django.contrib path.to.someotherapp

This will not run all tests from apps in INSTALLED_APPS automatically though. For a more complex solution, you could write your own runner, taking from both the old and new runner.

Also note that it usually shouldn't be necessary to run tests from django.contrib, as these are not testing your application, but rather the Django distribution. Django ships with even more tests, which are not run by either runner.

Community
  • 1
  • 1
Nicolas Cortot
  • 6,591
  • 34
  • 44
  • Ok... let me slightly modify the question. What is the 'best' way to run all tests using the new test runner? – Luke Jan 03 '14 at 14:04
  • @Luke you can pass `django.contrib` as an argument to the test command , as explained in my updated answer. But this will not run **all** apps. – Nicolas Cortot Jan 03 '14 at 14:15
  • Thanks, Nicolas. Though what I was trying to get at with my question is how to run all tests from all of my apps? running `python manage.py test myproject` continues to run 0 tests for me. Is this just my configuration? – Luke Jan 03 '14 at 14:40
  • `python manage.py test myproject` is right if `myproject` is a package in the current directory. Are your test modules named `test*.py`? Does the command `find myproject -name 'test*.py'` find your tests? – Nicolas Cortot Jan 03 '14 at 14:58
  • No. in my project directory I have: apps/app1/tests.py apps/app2/tests.py myproject/settings.py As I understand it, `manage.py test myproject` will look for test*.py under myproject directory. But it's not going to go and look for tests in each of my applications. what seems to be needed is someway of separating INSTALLED_APPS into two variables (MY_APPS, CONTRIB_APPS) and then passing the list of MY_APPS to the test runner? – Luke Jan 03 '14 at 16:20
  • The "standard" way with the new test runner would be to run `manage.py test` to run all tests (in the current dir), or `manage.py test apps`, `manage.py test apps.app1` etc. It seems you can also run `manage.py test . django.contrib` to run tests in the current dir *and* in `django.conrib`. – Nicolas Cortot Jan 03 '14 at 16:45
  • I discovered the root of my problem was having each app as `apps/app1`, `apps/app2`, etc, without a file called `__init__.py` in `apps/` - hence `apps` was not recognised as a module by the test runner. Brick wall vs head: 1-nil. – Luke Jan 06 '14 at 15:00
0

It's a shame that Django decided to ignore custom apps in the INSTALLED_APPS that are not in the project tree. See this post https://groups.google.com/forum/#!topic/django-users/gGfVhfrfE10 for their reasoning.

My real-world case does of course does not fall in the three mentioned use cases (big suprise!): We have a site that we deploy with a different collection of tightly coupled custom apps for client / group of clients. We don't want these apps nested under the project tree, because each is in its own git repo. In the past, we've used git submodules, fake submodules and subtrees; all had their issues in our setup. On the other hand, having each app as its own package on the same level as the site satisfies most of our requirements.

Of course each app has its own tests, but I would like to be able to run the full test suite (including the site and all custom apps) for each specific differently constituted site.

Our workaround is the following:

In settings/test.py I have:

ATA_BLACKLIST = ['scary_mod1', 'scary_mod2']
ADD_TEST_APPS = [i for i in INSTALLED_APPS
                 if '.' not in i and i not in ATA_BLACKLIST]
ATA_STR = " ".join(ADD_TEST_APPS)

I have a run_tests.sh script at top-level which looks more or less like this:

#!/bin/bash

MYDIR=`dirname $0`
cd $MYDIR
DJANGO_SETTINGS_MODULE=kmxng.settings.test
ATA_STR=`python -c "from django.conf import settings; print settings.ATA_STR"`
coverage run --omit "*lib/python*" ./manage.py test . $ATA_STR
coverage report

In short, in my test settings I generate a list of extra modules that should be added to testing. I invoke the django test command with that list in addition to the default ..

(I did have a look at overriding DiscoverRunner -- it has a relatively long build_suite method that could be overridden. The work-around above is a less-invasive option.)

Charl Botha
  • 4,373
  • 34
  • 53