How can I profile my python / django application which is crashing on a single request after 100 seconds of hogging more memory?
All I see in top is that the wsgi process is consuming memory slowly until it crashes.
The only profiling techniques I know run on a full request/response cycle but I'm not able to finish a request. What then?
I might even run the dev server and try to kill it mid-request and see where the stack is.
A bit fiddly and will have some overhead, but you could use sys.setprofile() to provide a function to be called on entry and exit from functions and dump progress of calls out to a log file yourself, potentially with check of memory in use at the same time.
http://docs.python.org/dev/library/sys.html#sys.setprofile
Also perhaps check out heapy as a way of getting console type access into your live process to dump out memory/object usage.
