What are best practices for adding initial configuration data to a Python Django instance?

Question

I have a Django application that has a Setting model. I've manually added the configuration data to the database through the Django Admin UI/Django Console but I want to package up this data and have it automatically created when an individual creates/upgrades an instance of this app. What are some of the best ways to accomplish this?

I have already looked at:

1. Django Migrations Topics Documentation which includes a section on Data Migrations
   • shows a data migration but it involves data already existing in a database being updated, not new data being added.
2. Django Migration Operations Reference
   • shows examples using RunSQL and RunPython but both only when adding a minimal amount of data.
3. How to create database migrations
   • looks at moving data between apps.
4. How to provide initial data for models
   • mentions data migrations covered in the previous docs and providing data with fixtures.

Still I haven't found one that seems to line up well with the use case I'm describing.

I also looked at several StackOverflow Q&As on the topic, but all of these are quite old and may not cover any improvements that occurred in the last 8+ years:

• Programmatically using Django's loaddata
• How can I provide the initial data to a Django model?
• Loading initial data with Django 1.7+ and data migrations

There is a decent amount of settings in this app and the above approaches seem pretty laborious as the amount of data increases.

I have exported the data (to JSON) using fixtures (e.g. manage.py dumpdata) but I don't want folks to have to run fixtures to insert data when installing the app.

It feels like there should be a really simple way to say, "hey, grab this csv, json, yaml, etc. file and populate the models."

Current Thoughts

Barring better recommendations from everyone here my thought is to load the JSON within a data migration and iterate through inserting the data with RunSQL or RunPython. Any other suggestions?

Answer 1

I would not recommend using database (directly) for accessing the settings. Because it will make the code messy. For example, for each settings, you have to call the database and you have to query like MyModel.objects.get(...). Even if you write a function to reduce repetitive code, it still won't reduce DB query.

Hence, I would recommend using libraries like django-constance if your settings are changing dynamically. In Django Constance, you can store the default configurations in settings.py (or a separate python file which can be imported in settings.py). Like this (copy pasted from documentation):

INSTALLED_APPS = (
    ...
    'constance',
)

CONSTANCE_CONFIG = {
    'BANNER': ("The national cheese emporium", 'Name of the shop '
                       'The national cheese emporium'),
    ...
}

The adminsite will look like this:

And access the variable in code:

from constance import config

print(config.BANNER)

Then you can modify the value in the admin site based on your need. The reason I recommend this way because the setting should be easily accessible from code and you can modify the settings from admin site dynamically. Also settings are configurable with custom fields and other features.

If there are pre-existing instances of this application which you want to update with Django-Constance, then I suggest writing a Django Management Command, which will sync the table from existing settings to the Django Constance table.

Answer 2

We can run function at the start of our django app. suppose we have a app named core, in the apps.py file we have

from django.apps import AppConfig

class CoreConfig(AppConfig):
    default_auto_field = 'django.db.models.BigAutoField'
    name = 'core'

    def ready(self):
        # your initial work at the start of app
        from core.utils import populate_initial_settings
        populate_initial_settings()

This will be run every time at the start of django server so you have to handle the logic accordingly. Like first counting the Setting objects if count = 0 populate Setting model otherwise not.

Answer 3

You can use post migrate management signal sent by django-admin, by connecting a receiver function to this signal on a specific application.

Although initial migrations are marked with an initial = True attribute, there seems to be no way to access this attribute using a signal. Nonetheless, this attribute and other information are present on the migration name e.g. 0001_initial.

So with an adaptation based on this answer we can access an application latest migration name and explore that fact.

Suppose I have an application named Core, and I want to automatically populate a few users from a .json file when first migrating:

users.json

[
    {"username": "admin", "password": "super_secret", "email": "admin@example.com", "is_staff": true, "is_superuser":true},
    {"username": "first_user", "password": "first_user_password", "is_staff": true, "email": "first_user@example.com"},
    {"username": "second_user", "password": "second_user_password", "email": "second_user@example.com"},
    {"username": "third_user", "password": "third_user_password", "email": "third_user@example.com"}
]

apps.py

from django.apps import AppConfig
from django.db.models.signals import post_migrate

class CoreConfig(AppConfig):
    default_auto_field = 'django.db.models.BigAutoField'
    name = 'core'

    def ready(self):
        # Implicitly connect signal handlers decorated with @receiver.
        from . import signals
        # Explicitly connect a signal handler.
        post_migrate.connect(signals.my_callback)

signals.py

import os
import json
from .models import User
from django.dispatch import receiver
from django.db.models.signals import post_migrate
from django.contrib.auth.hashers import make_password
from django.db.migrations.recorder import MigrationRecorder

@receiver(post_migrate)
def my_callback(sender, **kwargs):
    if sender.name == 'core':
        lastest_migration = MigrationRecorder.Migration.objects.filter(app=sender.name).order_by('-applied')[0]
        name = lastest_migration.name.split('_')

        if '0001' in name:
            if not User.objects.exists():
                file = open(os.getcwd() + '\\core\\data\\users.json')
                seed_data = json.load(file)
                file.close()
                
                for credential in seed_data:
                    credential['password'] = make_password(credential['password'])
                    user = User.objects.create(**credential)
                    print(f'created {user.username}...')
                print('finished seed...')

First, I used if 'initial' in name condition, but if you create a field with initial string on it then the migration will be named with it. If we really want to ignore everything else, maybe this is the best option. Or even the full default name 0001_initial.