6

The big picture is: I want to use eventlet in some application that does asynchronous IO while working with Django models externally. Working with Django externally is simple (see Django: How can I use my model classes to interact with my database from outside Django?) but it's not the main problem.

I presume (I haven't tested) that using Django from greenlets is dangerous. Actually, in the case of psycopg2, we have this warning (see http://www.initd.org/psycopg/docs/advanced.html#support-to-coroutine-libraries):

Psycopg connections are not green thread safe and can’t be used concurrently by different green threads

In Django there is one db connection per thread (right?) and as such it may lead to scary scenarios when used. Is it possible to provide a connection object manually somehow? Or make it "greenlet-local"?

My motivation is to use connection pool from eventlet (http://eventlet.net/doc/modules/db_pool.html) so that I could speed up my IO-bound application.

Community
  • 1
  • 1
thinred
  • 461
  • 3
  • 5

1 Answers1

8

There are some projects out there to make Django work well with greenlet. I would check out psycogreen which uses the coroutines support in Psycopg >= 2.2. There is a good blog post on using gevent, gunicorn, and psycogreen together with Django: http://dbinit.com/blog/going-green/.

ojrac
  • 13,231
  • 6
  • 37
  • 39
Mark Lavin
  • 24,664
  • 5
  • 76
  • 70
  • 1
    This is very helpful, but it does not answer my question precisely. After some reading and digging I'm pretty much convinced I can use Django from eventlet without any problem. The only thing to do is to run "eventlet.monkey_patch()" which will change threading.local (and other system libraries) and therefore connections will become "greenlet-local" instead of "thread-local". Additionally eventlet patches psycopg2 so everything should run smoothly. – thinred Mar 10 '11 at 08:30
  • Perhaps I misread this question. If your goal of using eventlet is for connection pooling why not use pgpool or pgbouncer? – Mark Lavin Mar 10 '11 at 12:04
  • I use eventlet to do some web-scraping stuff and I'm putting these data to database with Django models. Therefore I was curious if connections are "greenlet-safe" so that concurrent greenthreads won't run into each other. They are after monkey-patching. Also, after that, DB operations are concurrent as well. – thinred Apr 14 '11 at 10:20