Why isn't behave's documented manual integration with Django working?

Question

I have a Django (1.10.2) project ("theproject") and some behave (0.4.0) features. I've been using behave-django. python manage.py behave works. However, PyCharm (which uses the behave executable rather than a Django management command) doesn't know how to run my features, so I'm attempting to use behave's documented "manual integration" with Django.

My entire features/environment.py:

import os
import django
from django.test.runner import DiscoverRunner
from django.test.testcases import LiveServerTestCase
from splinter.browser import Browser

os.environ["DJANGO_SETTINGS_MODULE"] = "theproject.settings"

def before_all(context):
    django.setup()
    context.test_runner = DiscoverRunner()
    context.test_runner.setup_test_environment()
    context.old_db_config = context.test_runner.setup_databases()
    context.browser = Browser('phantomjs')

    # When we're running with PhantomJS we need to specify the window size.
    # This is a workaround for an issue where PhantomJS cannot find elements
    # by text - see: https://github.com/angular/protractor/issues/585
    if context.browser.driver_name == 'PhantomJS':
        context.browser.driver.set_window_size(1280, 1024)

def before_scenario(context, _):
    context.test_case = LiveServerTestCase
    context.test_case.setUpClass()

def after_scenario(context, _):
    context.test_case.tearDownClass()
    del context.test_case

def after_all(context):
    context.test_runner.teardown_databases(context.old_db_config)
    context.test_runner.teardown_test_environment()

    context.browser.quit()
    del context.browser

Here's INSTALLED_APPS from theproject/setting.py in case it's helpful (I removed 'behave-django' for this experiment):

INSTALLED_APPS = [
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django_extensions',
    'oauth2_provider',
    'push_notifications',
    'raven.contrib.django.raven_compat',
    'rest_framework',
    'app1.apps.App1Config',
    'app2',
    'django.contrib.admin'  # Must follow apps for apps' models to appear in admin UI
]

When I run behave I get

Exception AppRegistryNotReady: Apps aren't loaded yet.
Traceback (most recent call last):
  File "/usr/local/bin/behave", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/site-packages/behave/__main__.py", line 109, in main
    failed = runner.run()
  File "/usr/local/lib/python2.7/site-packages/behave/runner.py", line 672, in run
    return self.run_with_paths()
  File "/usr/local/lib/python2.7/site-packages/behave/runner.py", line 678, in run_with_paths
    self.load_step_definitions()
  File "/usr/local/lib/python2.7/site-packages/behave/runner.py", line 658, in load_step_definitions
    exec_file(os.path.join(path, name), step_module_globals)
  File "/usr/local/lib/python2.7/site-packages/behave/runner.py", line 304, in exec_file
    exec(code, globals, locals)
  File "features/steps/common.py", line 5, in <module>
    from django.contrib.auth.models import User
  File "/usr/local/lib/python2.7/site-packages/django/contrib/auth/models.py", line 4, in <module>
    from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "/usr/local/lib/python2.7/site-packages/django/contrib/auth/base_user.py", line 52, in <module>
    class AbstractBaseUser(models.Model):
  File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 105, in __new__
    app_config = apps.get_containing_app_config(module)
  File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 237, in get_containing_app_config
    self.check_apps_ready()
  File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 124, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

How can this way of integrating django and behave be made to work?

Something I tried that didn't work (or not completely): I moved django.setup() to the top level of environment.py, right after setting DJANGO_SETTINGS_MODULE. That fixes AppRegistryNotReady, but many scenarios fail with

IntegrityError: duplicate key value violates unique constraint "auth_user_username_key"
DETAIL:  Key (username)=(username) already exists.

Under behave-django a transaction was started before each scenario and rolled back afterwards; that seems not to be happening now. LiveServerTestCase extends TransactionTestCase, so I'm puzzled.

**AppRegistryNotReady: Apps aren't loaded yet.** This error erupts when the apps in INSTALLED_APPS settings are not in the right hierarchy. I suggest you check that. If the problem still persists then please share the INSTALLED_APPS settings. — Prakhar Trivedi, Dec 20 '16 at 12:51
Also, check answers of this question http://stackoverflow.com/questions/26276397/django-1-7-upgrade-error-appregistrynotready-apps-arent-loaded-yet and http://stackoverflow.com/questions/33186413/djangodjango-core-exceptions-appregistrynotready-apps-arent-loaded-yet — Prakhar Trivedi, Dec 20 '16 at 12:54
Thanks for the first suggestion, but my app runs, my django.test.TestCase tests pass, and, when I use when I use behave-django, my behave features run, so it seems like INSTALLED_APPS must be OK. Correct? — Dave Schweisguth, Dec 20 '16 at 12:56
Regarding those other questions: Most answers don't apply. The one suggesting `django.contrib.auth.get_user_model` didn't make any difference. I had already tried putting `django.setup()` at the top of `environment.py`; I'll add the unsuccessful result to my question. — Dave Schweisguth, Dec 20 '16 at 13:06
Dave, can you tell us the Django version you are using? Python is obviously 2.7. I've asked Andrey to look at this SO question via his blog post (comments). He is the original author of the integration published as "Manual Integration" in the Behave docs. — Peterino, Dec 20 '16 at 13:08
I still think that the error is somewhere in the settings.py file, Mainly INSTALLED_APPS may be. — Prakhar Trivedi, Dec 20 '16 at 13:09
@DaveSchweisguth Try this answer also http://stackoverflow.com/questions/25537905/django-1-7-throws-django-core-exceptions-appregistrynotready-models-arent-load — Prakhar Trivedi, Dec 20 '16 at 13:12
What `LiveServerTestCase` transaction management are you referring to? — knbk, Dec 20 '16 at 13:13
@PrakharTrivedi I don't see any applicable answers in that other question. I added INSTALLED_APPS to the question. — Dave Schweisguth, Dec 20 '16 at 13:23
I suspect that your database changes won't be rolled back between scenarios (hence the `IntegrityError`). The database is only torn down in `after_all()`. Try moving all code into the `before_scenario` and `after_scenario` functions, just for now to see if you can get things working. https://pythonhosted.org/behave/api.html#environment-file-functions -- Also, you may try to use `StaticLiveServerTestCase `, which is what we use with `behave-django`, but that's surely unrelated to your current issues. — Peterino, Dec 20 '16 at 13:26
@knbk `django.test.LiveServerTestCase` extends `django.test.TransactionTestCase`, which starts a transaction in `setUpClass` and rolls it back `tearDownClass`. The `IntegrityError`s suggest that that's not happening. — Dave Schweisguth, Dec 20 '16 at 13:30
Alright, this is something specific because I was able to run empty test feature the Python 2.7 with latest Django and the configuration above (phantom replaced by chrome). I can share my code somewhere. — Andrii Zarubin, Dec 20 '16 at 13:37
Here we go https://git.anvileight.com/andrey.zarubin/behavetest — Andrii Zarubin, Dec 20 '16 at 13:43
@Peterino moving `setup`/`teardown_databases` is sufficent to resolve the `IntegrityError`s. Doing `management.call_command('flush', verbosity=0, interactive=False)` also works and is faster. I'll probably need that level of database cleanup when I write features against async Javascript UIs (the features in this project happen to not touch any async Javascript), so that's likely to be my long term solution and you might want to write it up as an answer. It would still be good to know how to get transaction rollback to work. — Dave Schweisguth, Dec 20 '16 at 14:00
@AndreyZarubin thanks for taking a look. You'd need to import `django.contrib.auth.models.User` in a step file and create a `User` in a step (`User.objects.create_user('username'`) to see the issues I'm having. — Dave Schweisguth, Dec 20 '16 at 14:08
@AndreyZarubin yes (see my comment immediately before my previous response to you) and it works, although it doesn't use transactions (which would be faster) as the documented manual integration intends. — Dave Schweisguth, Dec 20 '16 at 14:14

score 2 · Accepted Answer · answered Dec 21 '16 at 20:57

Your database changes won't be rolled back between scenarios (hence the IntegrityError). The database is only torn down in after_all(). Try moving all code into the before_scenario and after_scenario functions (see related docs).

The execution time of your tests will increase, but the tests will be isolated. And as a quick fix this makes them pass for now at least. Hints for cleaner solutions in the comments and alternative answers.

score 1 · Answer 2 · answered Dec 20 '16 at 16:20

You seem to have misunderstood what TransactionTestCase (and LiveServerTestCase) does with relation to transactions. It's not a test case that uses transactions to run your tests and roll back changes at the end (this is what Django's "regular" TestCase does). It's a test case that does not use transactions to you can test your own transactions:

Django’s TestCase class is a more commonly used subclass of TransactionTestCase that makes use of database transaction facilities to speed up the process of resetting the database to a known state at the beginning of each test. A consequence of this, however, is that some database behaviors cannot be tested within a Django TestCase class. For instance, you cannot test that a block of code is executing within a transaction, as is required when using select_for_update(). In those cases, you should use TransactionTestCase.

(source)

TransactionTestCase still resets the database by flushing all tables, firing the post-migrate signal and reinstalling fixtures, but it runs under the assumption that __call__ is called on the test case. It seems that behave is calling run() directly, so it skips the setup/teardown of database data for each test.

You can probably fix this by creating your own subclass that calls _pre_setup() and _post_teardown() in setUp() and tearDown(). The only caveat is that every subclass that overrides setUp() or tearDown() must call super(), or _pre_setup()/_post_teardown() won't be called.

class BehaveLiveServerTestCase(LiveServerTestCase):
    def setUp(self):
        self._pre_setup()

    def tearDown(self):
        self._post_teardown()

It turns out that behave-django does call `__call__` on the test case, so I went with that. — Dave Schweisguth, Dec 22 '16 at 00:00

score 0 · Answer 3 · answered Dec 20 '16 at 13:51

0

I am pretty sure that the error is actually at

  File "features/steps/common.py", line 5, in <module>

Which tries to load the User before loading the apps. Try to move import inside step_impl.

answered Dec 20 '16 at 13:51

Andrii Zarubin

2,165
1
18
31

Thanks for the suggestion. That would fix `AppRegistryNotReady` at that point, but it would be unpleasant to have to import models in every step. `django.setup()` at the top level is less invasive. – Dave Schweisguth Dec 20 '16 at 14:04
Well, it depends on your philosophy :) I personally would prefer to have it in each step but up to you. – Andrii Zarubin Dec 20 '16 at 14:10

score 0 · Answer 4 · answered Dec 21 '16 at 23:58

Here's what I ended up with. It allows my features to run under the behave command without behave-django, and thus to run in PyCharm run configurations, without changing the features or steps. BehaviorDrivenTestCase and calling context.test() in before_scenario are copied from behave-django internals. I'm omitting browser setup and teardown and some other bits not relevant to this question.

import os
import django
from django.contrib.staticfiles.testing import StaticLiveServerTestCase
from django.core import management
from django.shortcuts import resolve_url
from django.test.runner import DiscoverRunner

os.environ["DJANGO_SETTINGS_MODULE"] = "api.settings"
django.setup()

def before_all(context):
    context.test_runner = DiscoverRunner()
    context.test_runner.setup_test_environment()
    context.old_db_config = context.test_runner.setup_databases()

def before_scenario(context, _):
    context.test = BehaviorDrivenTestCase()
    context.test.setUpClass()
    context.test() # this starts a transaction

    context.base_url = context.test.live_server_url
    def get_url(to=None, *args, **kwargs):
        return context.base_url + (
            resolve_url(to, *args, **kwargs) if to else '')
    context.get_url = get_url

class BehaviorDrivenTestCase(StaticLiveServerTestCase):
    """
    Test case attached to the context during behave execution

    This test case prevents the regular tests from running.
    """

    def runTest(*args, **kwargs):
        pass

def after_scenario(context, _):
    context.test.tearDownClass()
    del context.test

def after_all(context):
    context.test_runner.teardown_databases(context.old_db_config)
    context.test_runner.teardown_test_environment()

Isolating scenarios in transactions won't work for scenarios that exercise a UI that uses async Javascript. Removing context.test() and adding management.call_command('flush', verbosity=0, interactive=False) at the beginning of before_scenario also works, takes about the same amount of time, and should work better with async Javascript.

Why isn't behave's documented manual integration with Django working?

4 Answers4

Linked