0

django 1.3, uwsgi, python 2.7

I've got a view which is admitedly doing a number of large queries and inefficient stuffs. It is so bad that when the url is requested corresponding to that view, memory consumption jumps by like 200-300MBs.

I decided to try using pympler to try and pinpoint what was consuming so much memory. However inspecting various elements I can't figure out how ram consumption is jumping by 200-300MBs.

as the bulk of the stuff is happening in get_context_data function I've mainly been trying in here. but the largest object is like 11MBs. and even at the end when i run asizeof on the context variable it's only like 12-15MBs.

How do I determine why calling that view causes RAM usage to jump 200-300MBs?

w--
  • 6,427
  • 12
  • 54
  • 92
  • 1
    install django-debug-toolbar. It will give you an indication to the number & speed of your queries. – Timmy O'Mahony Mar 12 '13 at 23:20
  • 1
    can you post some of your code? – dm03514 Mar 12 '13 at 23:30
  • Make sure that in your settings.py DEBUG = False. – monkut Mar 12 '13 at 23:55
  • @TimmyO'Mahony thanks. already installed. but it doesn't tell me what is taking up all the RAM or am i just looking in the wrong place? – w-- Mar 13 '13 at 00:00
  • @dm03514 i could, but I'm more interested in what kinds of tools i can use to track this down. I feel this is a general thing i can apply all over the place. – w-- Mar 13 '13 at 00:04
  • @monkut yea i tested with DEBUG = False. – w-- Mar 13 '13 at 00:04
  • Try commenting one line per time of that view code to see which line is affecting performance the most. I would bet that you have a very inefficient query that is killing your ram.. When you find that line, add it to your question to see if we can help you – marianobianchi Mar 13 '13 at 03:41
  • The easiest way to reduce inefficient queries is getting comfortable with the [ORM (and optimization)](https://docs.djangoproject.com/en/1.1/topics/db/optimization/) and [Django Queries in general](https://docs.djangoproject.com/en/1.1/topics/db/queries/#querysets-are-lazy). You can try [profile for memory issues](http://stackoverflow.com/questions/487224/reducing-django-memory-usage-low-hanging-fruit) but once you are familiar with the ORM you'll probably be able to find the issues yourself intuitively – Timmy O'Mahony Mar 13 '13 at 08:53
  • 1
    Regarding django-debug-toolbar, it won't tell you how much RAM is consumed but it will A) show you the queries so you will be able to figure out which ones are returning a lot of rows (and where you can optimize accordingly) and B) how long is being spent on each query – Timmy O'Mahony Mar 13 '13 at 08:58

0 Answers0