How to skip an existing object instance when creating resources in bulk python

Question

I am trying to create a resources in bulk. While the resources are created I have the matric_no has to be unique. If the value of an existing matric_no is uploaded together with the some new entries, I get an integrity error 500 because the value already exists and it stops the rest of the values from being created. How can I loop through these values and then check if the value exists, and then skip so that the others can be populated? Here is my code:

**models.py**

from django.db import models
from django.utils.encoding import python_2_unicode_compatible

@python_2_unicode_compatible
class Undergraduate(models.Model):

    id = models.AutoField(primary_key=True)
    surname = models.CharField(max_length=100)
    firstname = models.CharField(max_length=100)
    other_names = models.CharField(max_length=100, null=True, blank=True)
    card = models.CharField(max_length=100, null=True, blank=True)
    matric_no = models.CharField(max_length=20, unique=True)
    faculty = models.CharField(max_length=250)
    department_name = models.CharField(max_length=250)
    sex = models.CharField(max_length=8)
    graduation_year = models.CharField(max_length=100)
    mobile_no = models.CharField(max_length=150, null=True, blank=True)
    email_address = models.CharField(max_length=100)
    residential_address = models.TextField(null=True, blank=True)
    image = models.CharField(max_length=250, 
    default='media/undergraduate/default.png', null=True, blank=True)

    def __str__(self):
        return "Request: {}".format(self.matric_no)


***serializers.py***

from .models import Undergraduate
from .models import Undergraduate

class UndergraduateSerializer(serializers.ModelSerializer):
    class Meta:
        model = Undergraduate
        fields ='__all__'


class CreateListMixin:    
    """Allows bulk creation of a resource."""    
    def get_serializer(self, *args, **kwargs):

        if isinstance(kwargs.get('data', {}), list):
            print(list)
            kwargs['many'] = True

        return super().get_serializer(*args, **kwargs)

**api.py**

from .models import Undergraduate

from rest_framework.viewsets import ModelViewSet

from .serializers import CreateListMixin,UndergraduateSerializer


class UndergraduateViewSet(CreateListMixin, ModelViewSet):

    queryset = Undergraduate.objects.all()
    serializer_class = UndergraduateSerializer
    permission_classes = (permissions.IsAuthenticated,)

**urls.py**

from rest_framework.routers import DefaultRouter
from .api import UndergradMassViewSet 
router=DefaultRouter() 

router.register(r'ug', UndergradMassViewSet)

This is the updated serializer.py

class UndergraduateSerializer(serializers.ModelSerializer):

    class Meta:
        model = Undergraduate
        fields = ('id', 'surname', 'firstname', 'other_names', 'card','matric_no', 'faculty', 'department_name', 'sex', 'graduation_year', 'mobile_no', 'email_address', 'residential_address')

        def create(self, validated_data):
            created_ids = []
            for row in validated_data:
                try:
                    created = super().create(row)
                    created_ids.append(created.pk)
                except IntegrityError:
                    pass
            return Undergraduate.objects.filter(pk__in=[created_ids])

This is how i moved it now Seriliazers.py

class UndergraduateSerializer(serializers.ModelSerializer):
    def create(self, validated_data):
        created_ids = []
        for row in validated_data:
            try:
                created = super().create(row)
                created_ids.append(created.pk)
            except IntegrityError:
            pass
        return Undergraduate.objects.filter(pk__in=[created_ids])

class Meta:
    model = Undergraduate
    fields = ('id', 'surname', 'firstname', 'other_names', 'card','matric_no', 'faculty', 'department_name', 'sex', 'graduation_year', 'mobile_no', 'email_address', 'residential_address')
    read_only_fields = ('id',)

When the list sent has an existing matric_no , it returns 500 ListSeriaizer is not iterable

Don't put the code in the Meta class. Put it directly in the serializer class — Ken4scholars, Jan 12 '19 at 13:07
@Ken4scholars Not yet, same error, ListSerializer object is not iterable — Chioma Igbokwe, Jan 13 '19 at 14:51
@ChiomaIgbokwe did you move the create method to the serializer? I tried it and it worked correctly. — Ken4scholars, Jan 13 '19 at 20:32

score 1 · Answer 1 · answered Jan 11 '19 at 10:28

You definitely have to implement a custom create() method in your serializer to handle such a case as the serializer's default create method expects one object.

Nonetheless, there is a couple of design decisions to consider here:

You can use bulk_create but it has a lot of caveats which are listed in the docs and it would still raise an integrity error since the inserts are done in one single commit. The only advantage here is speed
You can loop over each object and create them singly. This will solve the integrity issue as you can catch the integrity exception and move on. Here you'll lose the speed in 1
You can also check for integrity issues in the validate method before even moving on to create. In this way, you can immediately return an error response to the client, with information about the offending ro. If everything is OK, then you can use 1 or 2 to create the objects.

Personally, I would choose 2(and optionally 3). Assuming you also decide to chose that, this is how your serializer's create method should look like:

def create(self, validated_data):
    created_ids = []
    for row in validated_data:
        try:
            created = super().create(row)
            created_ids.append(created.pk)
        except IntegrityError:
            pass
    return Undergraduate.objects.filter(pk__in=[created_ids])

So in this case, only the created objects will be returned as response

I tried the custom create i still got 500 ListSerializer object is not iterable — Chioma Igbokwe, Jan 11 '19 at 15:24
Please post the code. I suspect you're trying to iterate over the serializer object instead of iterating over the validated_data — Ken4scholars, Jan 11 '19 at 15:35

How to skip an existing object instance when creating resources in bulk python

1 Answers1