Mysterious error when using django server with grequests

Question

Currently, I am running a vagrant server on Ubuntu 14.04 and I test all my django modules by using the simple python manage.py runserver 0.0.0.0:8000

Since I am connecting to the django webserver using chrome through http://localhost:8000 and the server is running on a VM, I am port forwarding through the usage of the following setting in Vagrantfile

config.vm.network "forwarded_port", guest: 8000, host: 8000

Everything runs normally (all modules/views/tests function as expected), however, ever since I started using grequests I get this weird error

Exception happened during processing of request from ('10.0.2.2', 63520)
Traceback (most recent call last):
  File "/home/vagrant/anaconda3/lib/python3.6/socketserver.py", line 639, in process_request_thread
    self.finish_request(request, client_address)
  File "/home/vagrant/anaconda3/lib/python3.6/socketserver.py", line 361, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/home/vagrant/anaconda3/lib/python3.6/socketserver.py", line 696, in __init__
    self.handle()
  File "/home/vagrant/anaconda3/lib/python3.6/site-packages/django/core/servers/basehttp.py", line 159, in handle
    self.raw_requestline = self.rfile.readline(65537)
  File "/home/vagrant/anaconda3/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
  File "/home/vagrant/anaconda3/lib/python3.6/site-packages/gevent/_socket3.py", line 385, in recv_into
    self._wait(self._read_event)
  File "/home/vagrant/anaconda3/lib/python3.6/site-packages/gevent/_socket3.py", line 157, in _wait
    self.hub.wait(watcher)
  File "/home/vagrant/anaconda3/lib/python3.6/site-packages/gevent/hub.py", line 651, in wait
    result = waiter.get()
  File "/home/vagrant/anaconda3/lib/python3.6/site-packages/gevent/hub.py", line 899, in get
    return self.hub.switch()
  File "/home/vagrant/anaconda3/lib/python3.6/site-packages/gevent/hub.py", line 630, in switch
    return RawGreenlet.switch(self)
gevent.hub.LoopExit: ('This operation would block forever', <Hub at 0x7f3b777e8af8 epoll pending=0 ref=0 fileno=34>)

Note that I am not using grequests and that simply importing it seems to cause this error even when it is not being called or anything

Anyone have any ideas?

score 3 · Accepted Answer · answered Apr 17 '17 at 04:10

3

This is an issue with one of the underlying dependency - gevent, which overrides the default behaviour of python builtin such as time, etc.

You would have to monkeypatch. Something like:

from gevent import monkey
monkey.patch_all()

Here is the relevant gevent documentation.

I recently ran into this exact issue - so stopped using grequests and implemented my own async requests logic

answered Apr 17 '17 at 04:10

karthikr

97,368
26
197
188

Hello, thanks for the response. Since grequests is used in multiple modules, would I basically need to call `monkey.patch_all()` under every grequests import then? Also, rolling your own async requests logic seems a bit harsh. Is there some major flaw that I dont know about? – AlanSTACK Apr 17 '17 at 05:00
Another followup, could you potentially point me to an alternative to grequests? I am currently just using it to handle multiple rest API calls at once (e.g. sending PATCH requests to certain APIs that only support editing one thing at a time) – AlanSTACK Apr 17 '17 at 07:33
I do not know a good alternative at the moment. You woukd have to build your own – karthikr Apr 18 '17 at 08:35
You need to patch only once, in *first* file in import chain (your entry point). If you have multiple independent entrypoints, do it in all of them. THe best way is to put these lines before any other import, right after `from __future__ import annotations` (or module docstring, if you don't use annotations)/ – STerliakov Sep 09 '22 at 20:04

K_D · Answer 2 · 2022-09-09T22:31:40.600

Wanted to post an update on this.

New docs on monkey patching are located here

I used this hook that Django added in 1.7 that I found here to run the import.

So in my app.py file I created an app config. (Realized and updated that this breaks the makemigration command, not sure if it will break anything else.)

from django.apps import AppConfig

class MyAppConfig(AppConfig):
        name = 'MyApp'
            def ready(self):
    if ('makemigrations' in sys.argv or 'migrate' in sys.argv):
        pass
    else:
        from gevent import monkey
        monkey.patch_all()

And in my settings.py file in INSTALLED_APPS[....] I added the config into that list.

INSTALLED_APPS = [
    'tracker.apps.MyAppConfig',
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'admin_extra_buttons',
]

Seems to have worked for me.

Does the order of `INSTALLED_APPS = [...]` matter in this case? — AlanSTACK, Sep 09 '22 at 22:52
That's something I haven't played around with yet, but I would suspect that it would be better for you to have your app first in the list to be loaded prior to the rest. — K_D, Sep 09 '22 at 23:06

Mysterious error when using django server with grequests

2 Answers2