Unable to migrate using ModelState and ProjectState using of migrations API in Django 3.0.3

Question

I am using ProjectState to migrate to a new attributes of a table. I am trying to understand the ModelState and ProjectState using of migrations API in Django 3.0.3.

I am unable to migrate to the new state which has new fields. Can someone help me with the ProjectState and ModelState usage of what to apply for new model_definition migration to work? The following code does not migrate to DB but doesnt give any error.

I want to migrate from a DB table state to another state and there are some metadata _meta.

• The current DB state model_state.fields is:

  [('id', <django.db.models.fields.AutoField>)]

• The future DB state model_state.fields after adding fields_attrs migrations should be this using the models_definition:

  [('id', <django.db.models.fields.AutoField>), ('name', <django.db.models.fields.CharField>)]

Model Definition Code is:

model_config object is

{
 '__module__': 'testmodule', 'app_label': 'testmodule', 
 '__unicode__': <function ModelScript.model_create_config.<locals>.<lambda> at 0x00000221B6FBEF70>, 
 'attrs': {'name': <django.db.models.fields.CharField>}
}

model_definition is:

model_definition = type(
                model_item.table_name,
                # TODO: Put this into Database
                # model_config.get("extends"),
                bases,
                model_config
            )

This is the code I am using:

from django.db.migrations.state import ProjectState
from django.db.migrations.migration import Migration
from django.db.migrations.state import ModelState
from django.db.migrations import operations

# model_definition is coming from a function as the following object
model_definition = {'__module__': 'testmodule', 'app_label': 'testmodule', '__unicode__': <function ModelScript.model_create_config.<locals>.<lambda> at 0x000002047275FF70>, 'attrs': {'name': <django.db.models.fields.CharField>}, '__doc__': 'SampleModel(id)', '_meta': <Options for SampleModel>, 'DoesNotExist': <class 'testmodule.SampleModel.DoesNotExist'>, 'MultipleObjectsReturned': <class 'testmodule.SampleModel.MultipleObjectsReturned'>, 'id': <django.db.models.query_utils.DeferredAttribute object at 0x00000204727F9430>, 'objects': <django.db.models.manager.ManagerDescriptor object at 0x00000204727F9490>}

model_state = ModelState.from_model(model_definition)

# field_attrs are all the new fields to be migrated         
for k,v in field_attrs.items():
    model_state.fields.append((k, v))

# Create a fake migration with the CreateModel operation
cm = operations.CreateModel(name=model_state.name, fields=model_state.fields)

migration = Migration("fake_migration", model_state.app_label)
migration.operations.append(cm)

# SHOULD ProjectState be used for the new definition to be APPLIED to DB and HOW?
state = ProjectState()
with db_conn.schema_editor(collect_sql=True, atomic=migration.atomic) as schema_editor:
     # Following create_model also doesnot migrate to Mysql DB
     # Gives a Table exists Error even with root user of mysql
     # schema_editor.create_model(model_definition)

     # Following doesnot migrate to the new required state
     state = migration.apply(state, schema_editor, collect_sql=True)
     # Following gives atomic transaction error if used along with atomic
     # following commit commented gives no error but doesnt migrate
     # db_conn.commit()

I have read this and using How to programmatically generate the CREATE TABLE SQL statement for a given model in Django?

Any help or resource is welcome on this.

Update: I did try the test cases of Django and it didn't work programmatically. Do I have to use addfield categorically? Unsure of how to get this working. Both projectstate and model_create way is not working

Answer 1

To start, you need to be using the model metaclass, ie. ModelBase, and not type:

from django.db.models.base import ModelBase

model_definition = ModelBase(
    model_item.table_name,
    bases,
    model_config
)

Once you use the proper metaclass, you will likely receive a myriad of errors, since you are using many class attributes that ModelBase sets internally, and is not expecting you to set yourself.

Instead of dumping all the attributes that your model has, you should only set the attributes that ModelBase expects to be set on a traditional model, which includes:

• __module__ and __qualname__
• model fields
• custom managers or querysets
• model methods
• model Meta

Everything else should be omitted.

So for example, if you have a models that look like this, in the module myapp.models:

class Parent(models.Model):
    name = models.CharField(max_length=45)

class Child(models.Model):
    name = models.CharField(max_length=45)
    parent = models.ForeignKey(Parent, on_delete=models.CASCADE)

class ModelWithMeta(models.Model):
    class Meta:
        db_table = 'some_table'

The dynamic version of these models need to look like this:

from django.db import models
from django.db.models.base import ModelBase

bases = (models.Model,)

Parent = ModelBase('Parent', bases, {
    '__module__': 'myapp.models',
    '__qualname__': 'Parent',
    'name': models.CharField(max_length=45),
})

Child = ModelBase('Child', bases, {
    '__module__': 'myapp.models',
    '__qualname__': 'Child',
    'name': models.CharField(max_length=45),
    'parent': models.ForeignKey('myapp.Parent', on_delete=models.CASCADE),
})

ModelWithMeta = ModelBase('ModelWithMeta', bases, {
    '__module__': 'myapp.models',
    '__qualname__': 'ModelWithMeta',
    'Meta': type('Meta', (), {'db_table': 'some_table'}),
})

I don't understand the purpose of your migration code, so I will assume that it was a hack in attempt to get the dynamic models working, which means you can probably throw it out altogether and use the builtin migration loader, ie:

python3 manage.py makemigrations myapp && python3 manage.py migrate myapp

I you aren't familiar with python metaclasses, I'd recommend reading up on them, since it's a prerequisite to understand my code.