44

I understand that Django, PyPy, and Psycopg2 all work correctly together, and speed.pypy.org claims great performance improvements over CPython. Are there any downsides?

jl6
  • 6,110
  • 7
  • 35
  • 65

2 Answers2

48

  1. Library support. Not all libraries are compatible with PyPy.

    Your best bet is to actually try running pypy manage.py test and see if it breaks. Then you know which dependencies need to be brought into line.

    http://reinout.vanrees.org/weblog/2011/06/06/django-and-pypy.html

  2. Webservers

    You can't use pypy with Apache. You need to use a pure-python webserver + nginx. You MAY get it working on uwsgi.

Thomas
  • 11,757
  • 4
  • 41
  • 57
  • 31
    I use pypy+ gunicorn + django + nginx and it works like a charm. – Games Brainiac Oct 12 '14 at 12:05
  • 3
    I ran pypy3 manage.py test and python3 manage.py test to compare their times (for reference, it ran about 100 tests). CPython was about 4 times faster than PyPy. PyPy is generally faster with pure Python code, but it cannot perform its optimizations with C code and ends up running slower. Things like password hashing, encryption, and JWT encoding are done using C libraries through an FFI. If your server isn't doing those things, PyPy might be faster. My server does ALL of those things, so PyPy is much slower. – tedtanner Oct 21 '21 at 19:42
13

The PyPy wiki lists Django as compatible, but it doesn't go into great detail about how much of Django was tested. I am not aware of any major Django deployment that runs PyPy instead of CPython. A better question is why you'd want to switch to PyPy for a Django app, especially as Django has been extensively tested and deployed with CPython.

PyPy is good for tasks that are computationally intensive. Web apps are usually not. The Django benchmark they base their performance numbers off is essentially a template rendering benchmark which is a CPU intensive task. This is not representative of most web apps where the bottle neck tends to be I/O. As such, PyPy may not speed up your site as much as those graphs lead you to believe.

CadentOrange
  • 3,263
  • 1
  • 34
  • 52
  • 74
    The "Webservers are IO Constrained, Don't worry about slow languages" fallacy is apparently alive and well. I know that 50% of the time spent in my database-heavy website IS ACTUALLY PYTHON PROCESSING TIME. Templates need rendering, querysets need deepcopying. Responsible developers should be looking for performance improvements everywhere, not just in the DB layer. – Thomas May 03 '13 at 04:12
  • 8
    "This is not representative of *most* web apps where the bottle neck tends to be I/O." As always you need to profile to figure out where your bottlenecks are. Nowhere does that imply that you needn't worry about writing optimised code. Responsible developers optimise the critical paths of their app. Switching to an experimental Python runtime based on a template rendering benchmark is hardly responsible! – CadentOrange May 03 '13 at 11:07
  • 5
    I did some tests of cpython2 vs pypy. Performance tests run much faster with pypy. On the other hand, django pages with db queries where served much faster with cpython. – francescortiz Jul 31 '13 at 10:50
  • 1
    I'm not saying the use of pypy with django is a good or bad (i'm here searching for an answer myself), but "Web apps are usually not" (computationally intensive) is a bit harsh. Why else would we need 12 core processor for one server?.. Why else would we need server-farms? in my opinion every bit of optimization counts, now, more then ever! – StefanNch May 03 '14 at 17:38
  • 2
    @StefanNch More cores == more concurrency == more concurrently served requests. Remember that "Premature optimisation is the root of all evil". Always profile, understand the problem then begin to think of the solution. – CadentOrange May 06 '14 at 08:58
  • 2
    @CadentOrange You are correct my friend, unfortunatelly, I have some Django-python sites that keeps me awake 20 hours/day. They are a couple years old (not premature at all) and the only conclusion I've got is that I really need to search for something faster (faster for image manipulation, faster for db operation, faster for everything). Django is "fast enough", but not for me. For me Django is really slow, so slow that not even (mem)caching helps. As a last resort, while I'm searching for something more server-friendly, I'm using a CDN to help me out a bit. – StefanNch May 06 '14 at 10:33
  • 3
    @StefanNch You have my sympathies, but it sounds like your site might be broken. Many high traffic sites run on Django. For instance, [Instagram](http://instagram-engineering.tumblr.com/post/13649370142/what-powers-instagram-hundreds-of-instances-dozens-of). While it's possible that your site is busier than Instagram, you need to profile your app to figure out why it's slow. A good start will be to install [Django debug toolbar](https://github.com/django-debug-toolbar/django-debug-toolbar) and go through the pages of your site that are slow. – CadentOrange May 06 '14 at 20:31
  • 1
    @Tomas , Agree 100%. python spend a lot of cpu when it comes to work with databases. What I currently observe is we have almost cpu bound web application. Yes : templates, lot of queries with internal caching, deepcopy, serialization, json processing. In my tests database and IO in a whole are just still in comparison to python processes. Less than 100Kb/s of IO where Mb/s bandwidth is possible. What is more interesting I tried to switch to pypy on this deployment and got almost double performance drop of the same code. – Zorg Jun 03 '15 at 16:03
  • 1
    @Thomas : 5 years late but there are two distinct reasons you'd want better performance. 1) To improve the user experience. On this the OP is right. You can make a django app respond to one web request with latency low enough that a human barely perceives it. 2) To decrease server costs. This is where you're right. You can make an order of magnitude saving on app servers compared to python. – Adamantish Jan 15 '21 at 09:07
  • @CadentOrange I don't think that Django debug toolbar is very helpfull. The timings are all completely off, for example my site loads in 6 seconds and shows 4 seconds of template render time with Debug toolbar enabled. It loads in 300ms without it enabled (not sure what percentage of that is template rendering). – timthelion Mar 20 '21 at 14:06