How to add through option to existing ManyToManyField with migrations and data in django

Question

I can't find reference to particular issue in docs or online.

I have an existing many to many relation.

class Books(models.Model):
    name = models.CharField(max_length=100)

class Authors(models.Model):
    name = models.CharField(max_length=100)
    books = models.ManyToManyField(Books)

This has migrations and data. Now I need to use through option in order to add one extra field in table holding many to many relation.

class Authorship(models.Model):
    book = models.ForeignKey(Books)
    author = models.ForeignKey(Authors)
    ordering = models.PositiveIntegerField(default=1)

class Authors(models.Model):
    name = models.CharField(max_length=100)
    books = models.ManyToManyField(Books, through=Authorship)

When I run migrations, django creates fresh migration for Authorship model. I tried to create migration file manually by adding ordering column in Authorship table and altering books column in Authors table but I get some migration issues.

operations = [
    migrations.AddField(
        model_name='authorship',
        name='ordering',
        field=models.PositiveIntegerField(default=1),
    ),
    migrations.AlterField(
        model_name='authors',
        name='books',
        field=models.ManyToManyField(to='app_name.Books', through='app_name.Authorship'),
    ),
]

When trying to migrate, it gives KeyError: ('app_name', u'authorship') I bet there are other things that are affected and thus errors.

What things am I missing? Is there any other approach to work with this?

Answer 1

There is a way to add "through" without data migrations. I managed to do it based on this @MatthewWilkes' answer.

So, to translate it to your data model:

1. Create the Authorship model only with book and author fields. Specify the table name to use the same name as the auto-generated M2M table you already have. Specify the unique_together attribute to match what the auto-generated M2M table does (source). Add the 'through' parameter.

    class Authorship(models.Model):
        book = models.ForeignKey(Books)
        author = models.ForeignKey(Authors)
   
        class Meta:
            db_table = 'app_name_authors_books'
            unique_together = ['book', 'author']
   
    class Authors(models.Model):
        name = models.CharField(max_length=100)
        books = models.ManyToManyField(Books, through=Authorship)
   

2. Generate a migration, but don't run it yet.

3. Edit the generated migration and wrap the migration operations into a migrations. SeparateDatabaseAndState operation with all the operations inside state_operations field (with database_operations left empty). You will end up with something like this:

    operations = [
        migrations.SeparateDatabaseAndState(state_operations=[
            migrations.CreateModel(
                name='Authorship',
                fields=[
                    ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                    ('book', models.ForeignKey(to='app_name.Books')),
                ],
                options={
                    'db_table': 'app_name_authors_books',
                },
            ),
            migrations.AlterField(
                model_name='authors',
                name='books',
                field=models.ManyToManyField(through='app_name.Authorship', to='app_name.Books'),
            ),
            migrations.AddField(
                model_name='authorship',
                name='author',
                field=models.ForeignKey( to='app_name.Author'),
            ),
            migrations.AlterUniqueTogether(
                name='authorship',
                unique_together={('book', 'author')},
            ),
        ])
    ]
   

4. You can now run the migration and add the extra ordering field to your M2M table.

Edit: Apparently, column names in the DB are generated slightly differently for automatic M2M tables as for models-defined tables. (I am using Django 1.9.3.)

After the described procedure, I also had to manually change the column names of a field with a 2-word name (two_words=models.ForeignKey(...)) from twowords_id to two_words_id.

Answer 2

Looks like there is no way to use through option without having to do data migrations. So had to resort to data migration approach, I took some ideas from @pista329's answer and solved the issue using following steps.

• Create Authorship model

    class Authorship(models.Model):
        book = models.ForeignKey(Books)
        author = models.ForeignKey(Authors)
        ordering = models.PositiveIntegerField(default=1)
  

• Keep original ManyToManyField relation, but add another field using above defined model as through model:

    class Authors(models.Model):
        name = models.CharField(max_length=100)
        books = models.ManyToManyField(Books)
        published_books = models.ManyToManyField(
            to=Books,
            through=Authorship,
            related_name='authors_lst' # different related name is needed.
        )
  

  IMPORTANT: You must use a different related_name to the existing ManyToManyField. If you don't do this then Django may lose the data in the original field.

• Add data migration to copy all data from old table to new Authorship table.

After this, books field on Authors model can be removed as we have new field named published_books.

Answer 3

Migrations can be messy sometimes.

If you want to alter m2m field with through, I would suggest to rename altered field Authors.books to Authors.book. When asked by makemigrations, if you changed name from books to book? [yN], choose "N" as No. Django will delete books and create book field instead of altering.

class Authorship(models.Model):
    book = models.ForeignKey("Books")
    author = models.ForeignKey("Authors")
    ordering = models.PositiveIntegerField(default=1)

class Authors(models.Model):
    name = models.CharField(max_length=100)
    book = models.ManyToManyField("Books", through="Authorship")

If you want to use books anyway, change book to books and repeat migration process with y as answer to makemigrations question about renaming.