0

I have a django site which has only an admin interface, since it's an internal thing.

Simple pages load almost instantly, like say a table with 10 items.

But a page with lots of inlines and so on will take 5-10 seconds to load, sometimes even longer. It's almost unusable.

This is NOT a huge database by any means. I have a few dozen models and no table has more than 500 entries.

Running Ubuntu 10.04 on a recent quad-core Xeon with 12 GB RAM.

The problems occur with the production server (mod_wsgi) and with the development server too. Generally worse with the production server, but both are seriously slow.

During the delay, one CPU core maxes out for apache (if I'm using the production server) or python (if I'm using the development server).

I have tried playing around with WSGIDaemonProcess, changing the number of threads and processes, but to no avail.

Below is my apache config file, but like I said I have problems with the dev server too so this may not really be an apache issue.

Thanks for any help. Apologies for any improper use of terminology, etc. I am a C and MATLAB programmer and don't really know much about servers, networks, databases or python. Also, I do realize there are several threads on slow django responses, and I have read them all, but I haven't found anything that helps my situation. Thanks!

WSGIDaemonProcess MY_ACTUAL_IP_ADDRESS user=MY_USERNAME group=MY_USERNAME threads=4
WSGIProcessGroup MY_ACTUAL_IP_ADDRESS

WSGIScriptAlias /wsgi /neuroling/projects/neuroxy/neuroxy_project/neuroxy.wsgi
<Directory /neuroling/projects/neuroxy/neuroxy_project/>
  Order allow,deny
  Allow from all
</Directory>

Alias /media/ /var/www/media/
Alias /static/ /var/www/static/

<Directory /var/www/static>
Order deny,allow
Allow from all
</Directory>

<Directory /var/www/media>
Order deny,allow
Allow from all
</Directory>
smwilsonau
  • 189
  • 1
  • 10
  • What do you mean by "lots of inlines?" Also, try opening the page, save the contents (Ctrl + S) as an HTML file on your desktop, then reopen from the desktop. If it is still slow, it is the browser, if it is fast, it is the server and we can further investigate. I'm sure its the server if the CPU is getting bogged, but just to be absolutely sure. – Furbeenator Feb 09 '12 at 17:25
  • On the slowest page type, there are six inlines, which typically have zero to two entries in them. But on another page that also takes seconds to load, there is just one inline, which typically has about 10 entries. I saved the web page as you suggested, and it can then be opened instantly with FireFox. So yep, it's the server, not the browser. Thanks for any help! – smwilsonau Feb 09 '12 at 20:21
  • Try to post at least some of the view code that sends the context back to the template. Maybe there are some embedded loops or some other parabolic code sequence. Just make sure there is no confidential information in it. – Furbeenator Feb 09 '12 at 21:05
  • I literally haven't written a line of code. I'm just using the admin interface. All I have written is models.py and admin.py. No loops or such... – smwilsonau Feb 09 '12 at 21:36
  • Oh, I misunderstood your question. So you are experiencing these slow page loads in the admin site, right? What part is slow, displaying your model's objects? You can use the Django shell to do some output of your objects to see if you still get the slowdown. Also, it is possible (but unlikely) it is the database. What database engine are you using? – Furbeenator Feb 09 '12 at 22:34
  • The slowness happens when I click a link through to the slow page. Initially, nothing happens, i.e. the browser stays on the referring page. The pointer wheel spins, and down the bottom it says "Waiting for ... Then after 5 or 10 or 15 seconds, it immediately loads the target page, no apparent delays once it starts. Does that help isolate the problem at all? By the way, I'm using postgresql. I'm pretty sure that's not the issue, because I do plenty of queries from the command line in psql and never had any problems. But you never know, I guess. Thanks for helping with this. – smwilsonau Feb 10 '12 at 02:04

2 Answers2

3

I have solved this now. I made a copy of my database and deleted most of the data, to see whether that would affect the speed. Indeed, it stopped being slow. That clued me into the fact that the problem was something about the database (or accessing it), rather than apache or anything like that.

So then I turned on logging in postgresql and looked at the logs. There were literally thousands of weird SQL statements getting sent that seemed irrelevant. Closer inspection revealed they was all being sent to fill in a ForeignKey select-box which contained thousands of entries. Only a few of those entries are actually possible in any given case, but it seems really complicated to filter select-boxes in django and I haven't accomplished that yet. But what I did was specify these fields as raw_id_fields, which prevents a select-box being created. That completely solved the slowness problem.

Thank you to those who gave suggestions on my question. I hope this answer may be helpful to others.

smwilsonau
  • 189
  • 1
  • 10
1

Have you tried Django debug toolbar to try and isolate the slow spot.

You could also use production application performance monitoring tools such as New Relic.

http://blog.newrelic.com/2011/11/08/new-relic-supports-python/

http://www.newrelic.com

New Relic has a two week trial period with all features available. During that time make good use of the performance breakdown and slow transaction trace features to work out the problem. After two weeks it will drop down to free subscription feature level and you will loose the slow transaction traces but will still have performance breakdowns against web transactions.

Graham Dumpleton
  • 57,726
  • 6
  • 119
  • 134
  • Ok, I'm trying to install the debug toolbar. I can't get it to work. I added this to settings.py: import sys path = '/...../django-debug-toolbar-944bfd8/debug_toolbar' if path not in sys.path: sys.path.append(path) and the code from the first answer here: [http://stackoverflow.com/questions/6390310/how-do-i-see-the-django-debug-toolbar](http://stackoverflow.com/questions/6390310/how-do-i-see-the-django-debug-toolbar) – smwilsonau Feb 09 '12 at 22:05
  • It is failing on the line 'debug_toolbar.middleware.DebugToolbarMiddleware', when I add that to the MIDDLEWARE_CLASSES... am I supposed to set the path or something? – smwilsonau Feb 09 '12 at 22:19