0

I have a query that is causing memory spikes in my application. The below code is designed to show a single record, but occasionally show 5 to 10 records. The problem is there are edge cases where 100,000 results are passed to MultipleObjectsReturned. I believe this causes the high memory usage. The code is:

try:
    record = record_class.objects.get(**filter_params)
    context["record"] = record

except record_class.MultipleObjectsReturned:
    records = record_class.objects.filter(**filter_params)
    template_path = "record/%(type)s/%(type)s_multiple.html" % {"type": record_type}
    return render(request, template_path, {"records": records}, current_app=record_type)

I thought about adding a slice at the end of the filter query, so it looks like this:

records = record_class.objects.filter(**filter_params)[:20]

But the code still seems slow. Is there a way to limit the results to 20 in a way that does not load the entire query or cause high memory usage?

Casey
  • 2,611
  • 6
  • 34
  • 60

1 Answers1

0

As this_django documentation says:

Use a subset of Python’s array-slicing syntax to limit your QuerySet to a certain number of results. This is the equivalent of SQL’s LIMIT and OFFSET clauses.

For example, this returns the first 5 objects (LIMIT 5):

Entry.objects.all()[:5]

So it seems that "limiting the results to 20 in a way that does not load the entire query" is being fulfiled .

So your code is slow for some other reason. or maybe you are checking the time complexity in wrong way.

Amrit
  • 2,115
  • 1
  • 21
  • 41
  • I did some memory profiling and the first try statement loads the entire query into memory! I am using Django 1.8, wondering if that is the issue. – Casey Feb 05 '18 at 16:36
  • as far as documentation says functioning of queryset have not changed from django 1.8 to 2.0 and if you look at https://stackoverflow.com/questions/6574003/django-limiting-query-results this link then what you may find is taht orm uses limit clause in background – Amrit Feb 06 '18 at 07:04