1

I have a django app I want to migrate to dotcloud. Many actions in Django internals and in my app are not asynchronous, i.e. they block the thread until they finish. When I was using Apache, that didn't pose a problem since a different thread is opened on every request. But it doesn't seem to be the case in nginx/uwsgi that dotcloud use. Seemingly, uwsgi has a --enable-threads and --threads options that can be used for multithreading, but:

  1. It is not clear what version of uwsgi dotcloud use, and if they support these features
  2. Since I have no one else asking about this, I was wondering if this is really the right way to get the concurrent requests running (using threads)
idanzalz
  • 1,740
  • 1
  • 11
  • 18
  • So wait, Django on nginx can only process one request at a time? Or am I misunderstanding? – acjay Apr 04 '13 at 15:44
  • 1
    Django does not handle multi-threading itself, it depends on the wsgi layer it uses (or anything above it). You can achieve different forms of concurrency (processes, threads, gthreads) by using different configurations of gunicorn or something similar above Django – idanzalz Apr 16 '13 at 07:39
  • I looked into a few days ago, and as far as I could tell, the uwsgi package Dotcloud is configured for by default uses Supervisord to run a number of uswsgi workers, which allow your app to serve more than one request at a time in parallel. Don't take that as the gospel though, because I'm not an expert in that layer. – acjay Apr 16 '13 at 21:58

3 Answers3

1

You could run Django with Gunicorn. Gunicorn, in turn, supports multiple worker classes, and people reported success running gunicorn+gevents+django together[1][2].

To use that on dotCloud, you will probably have to use dotCloud's custom service. If that's something that you want to try, I would personally start with dotCloud's reimplementation of python service using the custom service, and replace uwsgi with gunicorn in it.

Community
  • 1
  • 1
jpetazzo
  • 14,874
  • 3
  • 43
  • 45
1

I came here looking for some leads, which I found, thanks! There was a fair amount of leg work left to actually get stuff working, though.

Here is an example app on github that uses gunicorn, gevent, and socketio on dotcloud:

https://github.com/t1m0thy/django-tictactoe/tree/dotcloud

t1m0
  • 735
  • 1
  • 7
  • 14
-1

Threads is a problem in python - GIL doesn't allow them to run simultaneously. So multiprocessing is an answer.

Or you may take a look at gevent. Actually gevent is a kind of a hack (monkey patching of python stack) and so on, but it allows to launch green threads. I'm not sure if gevent can be combined with django, but google knows ;)

Alexander A.Sosnovskiy
  • 797
  • 7
  • 16
  • Python threads are "real" but because of the GIL, they have no benefit when you try to do a parallel computation that uses all cores or somethings like that. They are useful if you are waiting on IO. Moreover, this doesn't answer the question – idanzalz Aug 07 '12 at 20:42
  • gevent is useful if you are waiting on IO. Due to the Global Interpreter Lock, in CPython only one thread can execute Python code at once - read http://docs.python.org/library/threading.html – Alexander A.Sosnovskiy Aug 07 '12 at 20:44
  • yes, only one can "execute", but many can wait on IO so it works. And I don't think you can magically make django work asynchronously by integrating gevent into it, or at least I haven't heard of it. If you are aware of a way to do so, I will gladly reward you with all the points I can – idanzalz Aug 07 '12 at 21:02