1

I have the following code from a complex project:

from gevent import monkey
monkey.patch_all()
import gevent

import oursql

def run(num):
    conn = oursql.connect(host = ...)
    cursor = conn.cursor()
    cursor.execute('start transaction')
    for i in range(2):
        print num, i
        cursor.execute('UPDATE userobj SET timestamp=(timestamp + 1) WHERE id = 1')
        gevent.sleep()

    cursor.execute('rollback')


proc = [gevent.spawn(run, i) for i in range(2)]
gevent.wait(proc)

The engine is InnoDB and the output is:

0 0
1 0

Then the program hangs. I know the reason is mysql locks the row after the first greenlet executes the update statement, so the update in the other greenlet will block. But why does gevent not transfer control back to the first greenlet after the other one blocks on socket? And I wonder is there any graceful solution besides using a lock or committing before gevent.sleep?

p.s. The original situation is in a website project. I mixed pymongo and SQLAlchemy operations and used gunicorn to serve the site. But I found that parallel requests may block forever. After a long time of debug, I finally figured out this was due to pymongo used some socket operations, which caused gevent to switch to another greenlet, resulting a deadlock as the above code demonstrates.

Thanks!

jiakai
  • 501
  • 2
  • 14
  • Looks like oursql is implemented using Cython/C and uses libmysqlclient, so it won't be compatible with gevent and monkey patching (which patches Python libraries, not external C/C++ libraries). [This seems to confirm my suspicion](http://code.google.com/p/gevent/wiki/Compatibility). – robertklep Mar 21 '13 at 10:29
  • @robertklep yeah you are right:) I've seen that page before and see oursql in the replacement column, thought it should be compatible. But I've found just now that oursql is in fact oursql.so ... – jiakai Mar 21 '13 at 10:41
  • The page is rather confusing, but there's a comment towards the end of it stating that oursql is incompatible with gevent. – robertklep Mar 21 '13 at 10:43

1 Answers1

1

Thanks to @robertklep , one possible solution is to replace oursql with some pure-python drivers such as pymysql. In fact oursql is written in C and thus incompatible with gevent.

jiakai
  • 501
  • 2
  • 14