Optimizing queries in SerializerMethodField in Django REST framework

Question

I am accessing the related field's data in my SerializerMethodField and there is a query for every object that is being rendered. My models look like (will keep short for brevity):

class Listing(models.Model):

variant = models.ForeignKey(to='Variant', related_name='variant_listings')
seller = models.ForeignKey(to='user.Seller', related_name='seller_listings')
locality = models.ForeignKey(to='user.Locality', blank=True, null=True)
price = models.IntegerField(blank=True, null=True)

Variant, Seller and Locality are all related models.

My Viewset:

class ListingViewSet(viewsets.ModelViewSet):
    """Viewset class for Listing model"""

    queryset = Listing.objects.all()
    serializer_class = ListingSerializer
    pagination_class = TbPagination
    filter_backends = (filters.DjangoFilterBackend,)
    filter_class = ListingFilter

    def get_queryset(self):

        listing_qs = Listing.objects.filter(status='active')
        listing_qs = ListingSerializer.setup_eager_loading(listing_qs)
        return listing_qs

And my serializer:

class ListingSerializer(serializers.ModelSerializer):
"""Serializer class for Listing model"""

@staticmethod
def setup_eager_loading(queryset):
    queryset = queryset.prefetch_related('variant', 'seller', 'locality')
    return queryset

@staticmethod
def get_car_link(obj):
    variant_name_slug = obj.variant.name.replace(' ', '-').replace('+', '')
    return '/buy-' + obj.seller.city.name.lower() + '/' + variant_name_slug

car_link = serializers.SerializerMethodField(read_only=True)

@staticmethod
def get_car_info(obj):
    return {
        'id': obj.id,
        'variant_name': obj.variant.name,
        'localities': obj.locality.name,
    }

car_info = serializers.SerializerMethodField(read_only=True)

@staticmethod
def get_image_urls(obj):
    caption_ids = [1, 2, 3, 5, 7, 8, 18]
    attachments_qs = Attachment.objects.filter(listing_id=obj.id, caption_id__in=caption_ids)
    image_urls = []
    for attachment in attachments_qs:
        url = str(obj.id) + '-' + str(attachment.file_number) + '-360.jpg'
        image_urls.append(url)

    return image_urls

image_urls = serializers.SerializerMethodField(read_only=True)

class Meta:
    model = Listing
    fields = ('car_link', 'car_info', 'sort_by', 'image_urls')

For each listing returned by the listing viewset, there is a query for every related field accessed in the SerializerMethodField.

I found some related questions like this. But that didn't help. Also, I tried doing prefetch_related on my get_queryset method of the viewset and also implemented eager loading with the help of this article. But nothing helped.

Is there any way to avoid these queries?

Edit

The get_car_info function written above, contains a few more fields (along with the ones already present) which are required separately in a nested JSON by the name of car_info in the final serialized data that is being rendered at the front end.

`prefetch_related()` will help, but only for data you access through `obj`. You must not create other querysets as those will generate queries if evaluated. Soooo your attachments too must be prefetched somehow. Are you aware you can attach arbitrary querysets when using `prefetch_related`? — spectras, Jul 14 '17 at 08:40
We'll need to see how you're using `prefetch_related`. You are most likely using it incorrectly, or accessing the related data incorrectly. It is definitely the way forward. — Kye, Jul 14 '17 at 08:41
@Kye I have edited the code and added eager_loading in serializer and viewset. Please have a look. — Shubhanshu, Jul 14 '17 at 09:27
@spectras please see the modified code (eager loading in serializer and in get_queryset method of viewset). — Shubhanshu, Jul 14 '17 at 09:28

score 0 · Answer 1 · answered Mar 22 '18 at 09:34

I have used this article: http://ses4j.github.io/2015/11/23/optimizing-slow-django-rest-framework-performance/

I created a set up eager loading method in my serializer, like so:

class EagerGetProjectSerializer(serializers.ModelSerializer):
    lead_researcher = UserSerializer()
    participating_researcher = UserSerializer(many=True)
    client = ProjectClientSerializer()
    test_items = TestItemSerializer(many=True)


    @staticmethod
    def setup_eager_loading(queryset):
        queryset = queryset.select_related('lead_researcher', 'client')
        queryset = queryset.prefetch_related('participating_researcher', 
        'test_items')
        return queryset


    class Meta:
        model = Project
        fields = '__all__'

Notice that when referencing the objects you want to pull in the serializer you have to use the foreign key related name attribute.

and called in my view, before accessing the serializer:

class SingleProject(APIView):
    def get(self, request):
        ans = Project.objects.filter(id=project_id)
        qs = EagerGetProjectSerializer.setup_eager_loading(ans)
        serializer = EagerGetProjectSerializer(qs, many=True)

Optimizing queries in SerializerMethodField in Django REST framework

1 Answers1