Django data migration when changing a field to ManyToMany

Question

I have a Django application in which I want to change a field from a ForeignKey to a ManyToManyField. I want to preserve my old data. What is the simplest/best process to follow for this? If it matters, I use sqlite3 as my database back-end.

If my summary of the problem isn't clear, here is an example. Say I have two models:

class Author(models.Model):  
    author = models.CharField(max_length=100) 

class Book(models.Model):  
    author = models.ForeignKey(Author)  
    title = models.CharField(max_length=100)

Say I have a lot of data in my database. Now, I want to change the Book model as follows:

class Book(models.Model):  
    author = models.ManyToManyField(Author)  
    title = models.CharField(max_length=100) 

I don't want to "lose" all my prior data.

What is the best/simplest way to accomplish this?

Ken

Answer 1

I realize this question is old and at the time the best option for Data Migrations was using South. Now Django has its own migrate command, and the process is slightly different.

I've added these models to an app called books -- adjust accordingly if that's not your case.

First, add the field to Book and a related_name to at least one, or both of them (or they'll clash):

class Book(models.Model):  
    author = models.ForeignKey(Author, related_name='book')
    authors = models.ManyToManyField(Author, related_name='books')
    title = models.CharField(max_length=100) 

Generate the migration:

$ ./manage.py makemigrations
Migrations for 'books':
  0002_auto_20151222_1457.py:
    - Add field authors to book
    - Alter field author on book

Now, create an empty migration to hold the migration of the data itself:

./manage.py makemigrations books --empty
    Migrations for 'books':
0003_auto_20151222_1459.py:

And add the following content to it. To understand exactly how this works, check the documentation on Data Migrations. Be careful not to overwrite the migration dependency.

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations


def make_many_authors(apps, schema_editor):
    """
        Adds the Author object in Book.author to the
        many-to-many relationship in Book.authors
    """
    Book = apps.get_model('books', 'Book')

    for book in Book.objects.all():
        book.authors.add(book.author)


class Migration(migrations.Migration):

    dependencies = [
        ('books', '0002_auto_20151222_1457'),
    ]

    operations = [
        migrations.RunPython(make_many_authors),
    ]

Now remove the author field from the Model -- it should look like this:

class Book(models.Model):
    authors = models.ManyToManyField(Author, related_name='books')
    title = models.CharField(max_length=100)

Create a new migration for that, and run them all:

$ ./manage.py makemigrations
Migrations for 'books':
  0004_remove_book_author.py:
    - Remove field author from book

$ ./manage.py migrate
Operations to perform:
  Synchronize unmigrated apps: messages, staticfiles
  Apply all migrations: admin, auth, sessions, books, contenttypes
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying books.0002_auto_20151222_1457... OK
  Applying books.0003_auto_20151222_1459... OK
  Applying books.0004_remove_book_author... OK

And that's it. The authors previously available at book.author now should be in the queryset you get from book.authors.all().

Answer 2

Probably the best and easiest thing you should do would be:

Create the Many to many field with a different name say

authors = models.ManyToManyField(Author)

write a small function to convert foreignkey values to M2M values:

def convert():
    books = Book.objects.all()
    for book in books:
        if book.author:
            li = [book.author.id]
            book.authors.append(li)
            book.save()

Once it is run, you can delete the author field from the table and run migration again.