Duplicate Django Model Instance and All Foreign Keys Pointing to It

Question

I want to create a method on a Django model, call it model.duplicate(), that duplicates the model instance, including all the foreign keys pointing to it. I know that you can do this:

def duplicate(self):
   self.pk = None
   self.save()

...but this way all the related models still point to the old instance.

I can't simply save a reference to the original object because what self points to changes during execution of the method:

def duplicate(self):
    original = self
    self.pk = None
    self.save()
    assert original is not self    # fails

I could try to save a reference to just the related object:

def duplicate(self):
    original_fkeys = self.fkeys.all()
    self.pk = None
    self.save()
    self.fkeys.add(*original_fkeys)

...but this transfers them from the original record to the new one. I need them copied over and pointed at the new record.

Several answers elsewhere (and here before I updated the question) have suggested using Python's copy, which I suspect works for foreign keys on this model, but not foreign keys on another model pointing to it.

def duplicate(self):
    new_model = copy.deepcopy(self)
    new_model.pk = None
    new_model.save()

If you do this new_model.fkeys.all() (to follow my naming scheme thus far) will be empty.

Answer 1

You can create new instance and save it like this

def duplicate(self):
    kwargs = {}
    for field in self._meta.fields:
        kwargs[field.name] = getattr(self, field.name)
        # or self.__dict__[field.name]
    kwargs.pop('id')
    new_instance = self.__class__(**kwargs)
    new_instance.save()
    # now you have id for the new instance so you can
    # create related models in similar fashion
    fkeys_qs = self.fkeys.all()
    new_fkeys = []
    for fkey in fkey_qs:
        fkey_kwargs = {}
        for field in fkey._meta.fields:
            fkey_kwargs[field.name] = getattr(fkey, field.name)
        fkey_kwargs.pop('id')
        fkey_kwargs['foreign_key_field'] = new_instance.id
        new_fkeys.append(fkey_qs.model(**fkey_kwargs))
    fkeys_qs.model.objects.bulk_create(new_fkeys)
    return new_instance

I'm not sure how it'll behave with ManyToMany fields. But for simple fields it works. And you can always pop the fields you are not interested in for your new instance.

The bits where I'm iterating over _meta.fields may be done with copy but the important thing is to use the new id for the foreign_key_field.

I'm sure it's programmatically possible to detect which fields are foreign keys to the self.__class__ (foreign_key_field) but since you can have more of them it'll better to name the one (or more) explicitly.

Answer 2

Although I accepted the other poster's answer (since it helped me get here), I wanted to post the solution I ended up with in case it helps someone else stuck in the same place.

def duplicate(self):
    """
    Duplicate a model instance, making copies of all foreign keys pointing
    to it. This is an in-place method in the sense that the record the
    instance is pointing to will change once the method has run. The old
    record is still accessible but must be retrieved again from
    the database.
    """
    # I had a known set of related objects I wanted to carry over, so I
    # listed them explicitly rather than looping over obj._meta.fields
    fks_to_copy = list(self.fkeys_a.all()) + list(self.fkeys_b.all())

    # Now we can make the new record
    self.pk = None
    # Make any changes you like to the new instance here, then
    self.save()

    foreign_keys = {}
    for fk in fks_to_copy:
        fk.pk = None
        # Likewise make any changes to the related model here
        # However, we avoid calling fk.save() here to prevent
        # hitting the database once per iteration of this loop
        try:
            # Use fk.__class__ here to avoid hard-coding the class name
            foreign_keys[fk.__class__].append(fk)
        except KeyError:
            foreign_keys[fk.__class__] = [fk]

    # Now we can issue just two calls to bulk_create,
    # one for fkeys_a and one for fkeys_b
    for cls, list_of_fks in foreign_keys.items():
        cls.objects.bulk_create(list_of_fks)

What it looks like when you use it:

In [6]: model.id
Out[6]: 4443

In [7]: model.duplicate()

In [8]: model.id
Out[8]: 17982

In [9]: old_model = Model.objects.get(id=4443)

In [10]: old_model.fkeys_a.count()
Out[10]: 2

In [11]: old_model.fkeys_b.count()
Out[11]: 1

In [12]: model.fkeys_a.count()
Out[12]: 2

In [13]: model.fkeys_b.count()
Out[13]: 1

Model and related_model names changed to protect the innocent.

Answer 3

I tried the other answers in Django 2.1/Python 3.6 and they didn't seem to copy one-to-many and many-to-many related objects (self._meta.fields doesn't include one-to-many related fields but self._meta.get_fields() does). Also, the other answers required prior knowledge of the related field name or knowledge of which foreign keys to copy.

I wrote a way to do this in a more generic fashion, handling one-to-many and many-to-many related fields. Comments included, and suggestions welcome:

def duplicate_object(self):
    """
    Duplicate a model instance, making copies of all foreign keys pointing to it.
    There are 3 steps that need to occur in order:

        1.  Enumerate the related child objects and m2m relations, saving in lists/dicts
        2.  Copy the parent object per django docs (doesn't copy relations)
        3a. Copy the child objects, relating to the copied parent object
        3b. Re-create the m2m relations on the copied parent object

    """
    related_objects_to_copy = []
    relations_to_set = {}
    # Iterate through all the fields in the parent object looking for related fields
    for field in self._meta.get_fields():
        if field.one_to_many:
            # One to many fields are backward relationships where many child objects are related to the
            # parent (i.e. SelectedPhrases). Enumerate them and save a list so we can copy them after
            # duplicating our parent object.
            print(f'Found a one-to-many field: {field.name}')

            # 'field' is a ManyToOneRel which is not iterable, we need to get the object attribute itself
            related_object_manager = getattr(self, field.name)
            related_objects = list(related_object_manager.all())
            if related_objects:
                print(f' - {len(related_objects)} related objects to copy')
                related_objects_to_copy += related_objects

        elif field.many_to_one:
            # In testing so far, these relationships are preserved when the parent object is copied,
            # so they don't need to be copied separately.
            print(f'Found a many-to-one field: {field.name}')

        elif field.many_to_many:
            # Many to many fields are relationships where many parent objects can be related to many
            # child objects. Because of this the child objects don't need to be copied when we copy
            # the parent, we just need to re-create the relationship to them on the copied parent.
            print(f'Found a many-to-many field: {field.name}')
            related_object_manager = getattr(self, field.name)
            relations = list(related_object_manager.all())
            if relations:
                print(f' - {len(relations)} relations to set')
                relations_to_set[field.name] = relations

    # Duplicate the parent object
    self.pk = None
    self.save()
    print(f'Copied parent object ({str(self)})')

    # Copy the one-to-many child objects and relate them to the copied parent
    for related_object in related_objects_to_copy:
        # Iterate through the fields in the related object to find the one that relates to the
        # parent model (I feel like there might be an easier way to get at this).
        for related_object_field in related_object._meta.fields:
            if related_object_field.related_model == self.__class__:
                # If the related_model on this field matches the parent object's class, perform the
                # copy of the child object and set this field to the parent object, creating the
                # new child -> parent relationship.
                related_object.pk = None
                setattr(related_object, related_object_field.name, self)
                related_object.save()

                text = str(related_object)
                text = (text[:40] + '..') if len(text) > 40 else text
                print(f'|- Copied child object ({text})')

    # Set the many-to-many relations on the copied parent
    for field_name, relations in relations_to_set.items():
        # Get the field by name and set the relations, creating the new relationships
        field = getattr(self, field_name)
        field.set(relations)
        text_relations = []
        for relation in relations:
            text_relations.append(str(relation))
        print(f'|- Set {len(relations)} many-to-many relations on {field_name} {text_relations}')

    return self

Answer 4

Here is a somewhat simple-minded solution. This does not depend on any undocumented Django APIs. It assumes that you want to duplicate a single parent record, along with its child, grandchild, etc. records. You pass in a whitelist of classes that should actually be duplicated, in the form of a list of names of the one-to-many relationships on each parent object that point to its child objects. This code assumes that, given the above whitelist, the entire tree is self-contained, with no external references to worry about.

One more thing about this code: it is truly recursive, in that it calls itself for each new level of descendants.

from collections import OrderedDict

def duplicate_model_with_descendants(obj, whitelist, _new_parent_pk=None):
    kwargs = {}
    children_to_clone = OrderedDict()
    for field in obj._meta.get_fields():
        if field.name == "id":
            pass
        elif field.one_to_many:
            if field.name in whitelist:
                these_children = list(getattr(obj, field.name).all())
                if children_to_clone.has_key(field.name):
                    children_to_clone[field.name] |= these_children
                else:
                    children_to_clone[field.name] = these_children
            else:
                pass
        elif field.many_to_one:
            if _new_parent_pk:
                kwargs[field.name + '_id'] = _new_parent_pk
        elif field.concrete:
            kwargs[field.name] = getattr(obj, field.name)
        else:
            pass
    new_instance = obj.__class__(**kwargs)
    new_instance.save()
    new_instance_pk = new_instance.pk
    for ky in children_to_clone.keys():
        child_collection = getattr(new_instance, ky)
        for child in children_to_clone[ky]:
            child_collection.add(duplicate_model_with_descendants(child, whitelist=whitelist, _new_parent_pk=new_instance_pk))
    return new_instance

Example usage:

from django.db import models

class Book(models.Model)

class Chapter(models.Model)
    book = models.ForeignKey(Book, related_name='chapters')

class Page(models.Model)
    chapter = models.ForeignKey(Chapter, related_name='pages')

WHITELIST = ['books', 'chapters', 'pages']
original_record = models.Book.objects.get(pk=1)
duplicate_record = duplicate_model_with_descendants(original_record, WHITELIST)