I'm creating an app in several different python web frameworks to see which has the better balance of being comfortable for me to program in and performance. Is there a way of reporting the memory usage of a particular app that is being run in virtualenv?
If not, how can I find the average, maximum and minimum memory usage of my web framework apps?
It depends on how you're going to run the application in your environment. There are many different ways to run Python web apps. Recently popular methods seem to be Gunicorn and uWSGI. So you'd be best off running the application as you would in your environment and you could simply use a process monitor to see how much memory and CPU is being used by the process running your applicaiton.
I'll second Matt W's note about the application environment being a major factor ( Gunicorn , uWSGI, nginx->paster/pserve, eventlet, apache+mod_wsgi, etc etc etc)
I'll also add this- the year is 2012. In 1999, memory and CPU for stuff like this were huge concerns. But it's 2012. Computers are significantly more powerful, expanding them is much easier and cheaper, and frameworks are coded better.
You're essentially looking at benchmarking things that have no practical matter and will only be theoretically 'neat' and informative.
The performance bottlenecks on Python webapps are usually:
In terms of database communications bottleneck , the general approaches to solving it are:
In terms of database schema, convenience comes at a price. It's faster to get certain things done in Django -- but you're going to be largely stuck with the schema it creates. Pyramid+SqlAlchemy is more flexible and you can build against a finely tuned database with it... but you're not going to get any of the automagic tools that Django gives.
for concurrent connections / requests per second, it's largely due to the environment. running the same app under paster, uwsgi and other deployment strategies will have different results.
here's a link to a good , but old, benchmark - http://nichol.as/benchmark-of-python-web-servers
you'll note there's a slide for peak memory usage there, and although there are a few outliers and a decent amount of clustering going on, the worst performer had 122MB. that's nothing.
you could interpret gevent as awesome for having 3MB compared to uwsgi's 15 or cogen's 122... but these are all a small fraction of a modern system's memory.
the frameworks have such a small overhead and will barely be a factor in operating performance. even the database portions are nothing. reference this posting about SqlAlchemy ( Why is SQLAlchemy insert with sqlite 25 times slower than using sqlite3 directly? ) , where the maintainer notes some impressive performance notes: straight-up sql generation was ~.5s for 100k rows. when a full ORM with integrity checks/etc is involved , it becomes 16s for the same amount of rows. that is nothing.
So , my point is simple- the two factors you should consider are:
play with the frameworks to decide which one you like the most, but don't waste your time on performance testing , because all you're going to do is waste time.
The choice of hosting mechanism isn't the cause of memory usage, it is how you configure them, plus what fat Python web application you decide to run.
The benchmark being quoted of:
http://nichol.as/benchmark-of-python-web-servers
is a good example of where benchmarks can get it quite wrong.
The configurations of the different hosting mechanisms in that benchmark were not comparable and so there is no way you can use the results to evaluate memory usage of each properly. I would not pay much attention to that benchmark if memory is your concern.
Ignoring memory, some of the other comments made about where the real bottlenecks are going to be are valid. For a lot more detail on this whole issue see my PyCon talk.
