How to ensure fixture getting loaded only once for all testcases in django 1.9?

Question

I'm working on test cases for my project where I've to use fixture for data. The fixture is quite big (around 20 MB) for entire project. Since the project involves multiple apps so I want that the fixture gets loaded only once when I run tests. Specifying fixtures on each test cases as well as using keepdb flag is not working great and it takes alot of time to all tests. Can you please suggest a good workaround for the same so I can speed it up. I also followed this post but its not working with django 1.9.

Added sample tests App1/tests.py

from django.test import TestCase
class App1View1TestCase(TestCase):
    fixtures = [
        'fixtures/app_fixture.json',
    ]
class App1View2TestCase(TestCase):
    fixtures = [
        'fixtures/app_fixture.json',
    ]
class App1View3TestCase(TestCase):
    fixtures = [
        'fixtures/app_fixture.json',
    ]

App2/tests.py

from django.test import TestCase
class App2View1TestCase(TestCase):
    fixtures = [
        'fixtures/app_fixture.json',
    ]
class App2View2TestCase(TestCase):
    fixtures = [
        'fixtures/app_fixture.json',
    ]
class App2View3TestCase(TestCase):
    fixtures = [
        'fixtures/app_fixture.json',
    ]

Each testcase has more than 5 tests.

Have you read [the docs](http://pytest.org/latest/fixture.html)? The section on scopes seems relevant. — jonrsharpe, Jul 03 '16 at 18:50
hey @jonrsharpe I'm new to django testing and trying to stick only to that. I'm not familiar with pytest and django. did you ever used both on a project? — Tarun Behal, Jul 03 '16 at 18:54
Oh, you don't mean pytest fixtures? Sorry, my fault; could you clarify what you do mean? A [mcve] would be helpful. — jonrsharpe, Jul 03 '16 at 18:55
@TarunBehal Have you considered trying with Nose? It appears the django test runner might not be flexible straightforward to do that. See here: http://stackoverflow.com/a/9816748/1757321 — KhoPhi, Jul 03 '16 at 18:55
@Rexford nopes I didnt. The issue is I cant introduce a new app myself. In case there is no way that django test runner can handle that, then I'll see for this option. Do you prior experience on this? Did you faced any limitation with it? — Tarun Behal, Jul 03 '16 at 18:59
@TarunBehal "you can't introduce a new app" yourself? I understand you want a fixture to load once for entire test cases to run. Try nose. If what test to run or what app to test is what you're wondering about, then I'll recommend you consider the django docs more. — KhoPhi, Jul 03 '16 at 19:02
@Rexford the issue with introducing new app will involve clients approval and then endless discussion. "If what test to run or what app to test" I can run individual tests. Issue comes when build gets created. Since it triggers `python manage.py test` command so at that point all tests of the project gets executed and then it takes a lot of time to finish. I'll wait if I can be pointed to a direction using django or some patch on django test runner. Else I'll start exploring django nose. thanks for help.. :) — Tarun Behal, Jul 03 '16 at 19:08
@TarunBehal you are in luck! Will answer your question shortly — e4c5, Jul 04 '16 at 12:05

score 1 · Answer 1 · edited May 29 '18 at 13:46

Speed up fixture loading.

First you need the fixture as a CSV. If you already have a CSV serializer installed, you are in luck. Or connect with psql or pgadmin and create a CSV dump and skip the next bit.

To create a CSV from your existing fixture, just one of your tests, with the -k flag to manage.py test, which is the same as --keepdb, something like this:

 ./manage.py test -k myapp.test.App2View1TestCase.test_something

Now, you have the test database created with all your tables.

Add an entry to settings.py DATABASES section pointing to the test datbase (usually your databasename prefixed with test). Now manage.py loaddata from your fixture. Then you need to open your psql console and dump that data as CSV with COPY TO. you need to do that for each table.

I know this sounds like a lot of work, but you only need to do it once. But OK, how do we load this? Override the setUpTestData method and add some code like this:

@classmethod
def setUpTestData(cls):
    cursor = connection.cursor()
    cursor.execute("COPY table(fields) FROM dump_file.csv")

This you will find is a lot faster than using loaddata (as done by fixtures = [] ) and, once the fixture is loaded, it will be used by all the tests in the class.

Using the same fixture for all the testcases

Instead of overriding setUpTestData, override setUpClass

@classmethod
def setUpClass(cls):

    if not MyModel.objects.count():
        cursor = connection.cursor()
        cursor.execute("COPY table(fields) FROM dump_file.csv")

    super(TestCase, cls).setUpClass()

Disclaimer: I haven't used this method myself.

thanks alot for your effort.. I'll try this tomorrow and will update you. — Tarun Behal, Jul 04 '16 at 15:39

How to ensure fixture getting loaded only once for all testcases in django 1.9?

1 Answers1

Speed up fixture loading.

Using the same fixture for all the testcases