django fixtures (from dumpdata) failing when testing

Question

I'm trying to dump data from my production server to use as test on my dev server, but I'm getting errors when running "./manage.py test" on the dev server specifing the fixture file created on the prod server.

Here are the tries I made based on google/stackoverflow search:

# python manage.py dumpdata --indent=4 --natural
error when running tests: IntegrityError: (1062, "Duplicate entry 'cms-agencies' for key 'app_label'")

# python manage.py dumpdata --exclude contenttypes --indent=4
error when running tests: IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`test_current`.`django_admin_log`, CONSTRAINT `content_type_id_refs_id_288599e6` FOREIGN KEY (`content_type_id`) REFERENCES `django_content_type` (`id`))')

# python manage.py dumpdata --exclude contenttypes --natural --indent=4
error when running tests: IntegrityError: (1062, "Duplicate entry '14-add_agencies' for key 'content_type_id'")

# python manage.py dumpdata --exclude contenttypes --exclude auth --indent=4
error when running tests: IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`test_current`.`django_admin_log`, CONSTRAINT `user_id_refs_id_c8665aa` FOREIGN KEY (`user_id`) REFERENCES `auth_user` (`id`))')

# python manage.py dumpdata --exclude contenttypes --exclude auth --natural --indent=4
error when running tests: IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`test_current_abril`.`django_admin_log`, CONSTRAINT `user_id_refs_id_c8665aa` FOREIGN KEY (`user_id`) REFERENCES `auth_user` (`id`))')

I also tried removing "'init_command': 'SET storage_engine=INNODB'" from settings.py and still got 1062 errors.

I don't understand this problem. Shouldn't django just recreate the DB exactly as it was on the prod server when I load the fixtures?

agreed, would love an option to just do an exact copy of the db — Michael Bylstra, Oct 04 '12 at 05:25

score 14 · Answer 1 · edited May 23 '17 at 12:17

14

I had similar problems and these args worked for me:

python manage.py dumpdata --natural --exclude auth.permission --exclude contenttypes --indent 4

I also had a lot of issues with the post_save signal creating objects twice. There's a fix for that: How do I prevent fixtures from conflicting with django post_save signal code?

edited May 23 '17 at 12:17

Community

1
1

answered Oct 04 '12 at 05:19

Michael Bylstra

5,042
4
28
24

3

Thanks! `--natural` has been decrecated in Django 1.9, replacement is `--natural-foreign`. See [the documentation](https://docs.djangoproject.com/en/1.11/ref/django-admin/#dumpdata) for details. – mhkuu Sep 19 '17 at 11:48

score 1 · Answer 2 · answered Apr 27 '12 at 12:49

I believe the errors are telling you exactly what is happening. Is app_label unique? I would guess it is. I would think you have two objects with the same app_label key value. (cms-agencies)

Additionally when you have a foreign key relationship you need to have the object that corresponds to the foreign key. Dump data doesn't do this because it only dumps one model.

Something like https://github.com/davedash/django-fixture-magic is great for this. It dumps your model AND all fk dependencies.

In the fixture, yes. What I'm guessing is happening is that django adds some data to the DB (of its own models), and then it conflicts with the fixture — Lem0n, Apr 27 '12 at 13:09

score 1 · Answer 3 · answered Mar 31 '16 at 23:06

The problem is, that if you use natural keys (natural-foreign keys in later versions of Django), Django will actually store many-to-many relationships inside parent objects. That is something you want. But then you cannot simply dump all tables, you must not include the many-to-many tables/models in your dump as you would then load same data twice - and boom, duplicates and IntegrityErrors.

For example, you should dump auth.User and auth.Group, but not auth.User_Groups. Look at example of a dump from Django 1.7:

{
    "model": "auth.group",
    "fields": {
        "permissions": [],
        "name": "ST"
    },
},
{
    "model": "auth.group",
    "fields": {
        "permissions": [],
        "name": "property_manager"
    },
},

{
    "model": "auth.user",
    "fields": {
        "username": "boss",
        "groups": [
            ["property_manager"],["ST"],
        ],
        "user_permissions": [],
    },
},

Following line creates a comprehensive dump of users/permissions and content types you can move to dev to get identical copy, including identical row order and primary keys (tested on Django 1.7):

python manage.py dumpdata auth.User auth.Group contenttypes auth.Permission --indent 4 --natural-foreign > users.json

score 0 · Answer 4 · answered Oct 04 '12 at 05:36

0

your problem might be that you dumped out a unicode file but django expects a ascii file format when loading. This will happen using PowerShell on Windows if that applies to you.

answered Oct 04 '12 at 05:36

Nick

1,148
2
14
28

django fixtures (from dumpdata) failing when testing

4 Answers4