Limiting the amount of fixtures in django dumpdata

Question

One of our servers has 4 gb of data. But as of now i am only interested in populating only little data for fixtures. One easy of dumping data is:

 python manage.py dumpdata --indent=4 > shipping_fixture.json

but the trouble with this is that it dumps all the data into the database. Working with such a massive amount of data on the test does not make any sense. Is there any way out where i can limit the amount of data that does not make things heavy for me, and the data i downloaded is complete in itself.

Answer 1

The latest django (1.5.5) doesn't have such an option, but I think it's coming up soon in a future version. There is currently a ticket implementing a new feature to the dumpdata command that will allow you to filter what gets outputted based on the primary key if the model is specified.

A 3rd party app called django-test-utils can probably do what you need.

Use Django dumpdata to dump a subset of overall data?

Answer 2

I would recommend to use django serialization [1]. It helps to dump a custom query into a data file.

To dump the data:

from django.core import serializers

with open("/tmp/file.json", "w") as f:
    serializers.serialize('json', query, stream=f)

And to load the data:

with open("file.json", "r") as file:
    data = file.read()

for obj in serializers.deserialize("json", data):
    do_something_with_the_object(obj)

[1] https://docs.djangoproject.com/en/dev/topics/serialization/