How do you find the CPU consumption for a piece of Python?

Question

Background

I have a Django application, it works and responds pretty well on low load, but on high load like 100 users/sec, it consumes 100% CPU and then due to lack of CPU slows down.

Problem:

Profiling the application gives me time taken by functions.
This time increases on high load.
Time consumed may be due to complex calculation or for waiting for CPU.

So, how to find the CPU cycles consumed by a piece of code ?

Since reducing the CPU consumption will increase the response time.

I might have written extremely efficient code and need to add more CPU power

OR

I might have some stupid code taking the CPU and causing the slow down ?

Update

I am using Jmeter to profile my web app, it gives me a throughput of 2 requests/sec. [ 100 users]
I get a average time of 36 seconds on 100 request vs 1.25 sec time on 1 request.

More Info

Configuration Nginx + Uwsgi with 4 workers
No database used, using a responses from a REST API
On 1st hit the response of REST API gets cached, therefore doesn't makes a difference.
Using ujson for json parsing.

Curious to know:

Python-Django is used by so many orgs for so many big sites, then there must be some high end Debug / Memory-CPU analysis tools.
All those I found were casual snippets of code that perform profiling.

My first step in profiling is to see *which function(s)* most of the time is spent in, and they investigating why, if it's justified, and if/how it can be reduced. The *amount* of time (often) isn't as important as the *proportional amount* of time. — , Jun 04 '12 at 06:19
All the time taken seems to be distributed throughout the application. No specific point seem to be a problem area. — Yugal Jindle, Jun 04 '12 at 06:21
Seems? Did you exact measurements? See http://stackoverflow.com/a/582337/851737 for help. — schlamar, Jun 04 '12 at 06:28
I am using : http://djangosnippets.org/snippets/186/ for profiling my stuff. — Yugal Jindle, Jun 04 '12 at 06:38
BTW, what server (Apache+WSGI) are you running it on? How many **processes** does your Django app get? And is it mostly DB-bound or CPU-bound? — Jesvin Jose, Jun 04 '12 at 06:40
Do you have total throughput of 2 reqs/sec or 2*100 reqs/sec? — Lycha, Jun 04 '12 at 08:12
Do you have debug turned on? Django's DEBUG mode adds considerable overhead. What caching engine are you using? same machine or different machine? — Thomas, Jun 04 '12 at 08:38

score 2 · Accepted Answer · answered Jun 05 '12 at 23:40

You could try configuring your test to ramp up slowly, slow enough so that you can see the CPU gradually increase and then run the profiler before you hit high CPU. There's no point trying to profile code when the CPU is maxed out because at this point everything will be slow. In fact, you really only need a relatively light load to get useful data from a profiler.

Also, by gradually increasing the load you will be better able to see if there is a gradual increase in CPU (suggesting a CPU bottleneck) or if there is a sudden jump in CPU (suggesting perhaps another type of problem, one that would not necessarily be addressed by more CPU).

Try using something like a Cosntant Throughput Timer to pace the requests, this will prevent JMeter getting carried away and over-loading the system.

Good advise for a start.. I will explore the options you suggested. Apart from that, I am looking more of an ninja profiling tool for django / python - That can provide a more granular look into the system. :) — Yugal Jindle, Jun 06 '12 at 04:56

score 0 · Answer 2 · answered Sep 08 '12 at 11:58

0

Checkout New Relic for some pretty sweet analytics, they've got django-specific logging.

answered Sep 08 '12 at 11:58

Jens Alm

3,027
4
22
24

How do you find the CPU consumption for a piece of Python?

Background

Update

2 Answers2