0

I'd like to use requests 's session to reuse connections in django.

Reusing connections in Django with Python Requests

says I only need to declare it in global and access it.
However I doubt it is working as expected because it didn't get any faster in my test.

Is there a way to see if connection is actually reused as described here?

http://docs.python-requests.org/en/latest/user/advanced/

Django spawns separate thread for each requests and I think it defeats the mechanism to share the connection. (because session won't be shared across multiple threads) .. this is my hypothesis..

Community
  • 1
  • 1
eugene
  • 39,839
  • 68
  • 255
  • 489
  • Django doesn't spawn new threads. The WSGI server that you run Django will or will not spawn threads and even new subprocesses to handle incoming requests. Check your WSGI server configuration and documentation. – Martijn Pieters Jan 13 '16 at 12:09
  • 1
    The `requests` package uses a *thread safe shared connection pool*, so wether or not threads are used doesn't matter here, a connection can still be reused even if separate threads try to reach the same host *sequentially*. Otherwise, extra connections will be opened to run HTTP requests in parallel. – Martijn Pieters Jan 13 '16 at 12:12
  • @MartijnPieters: I have `enable-threads = true` in my uwsgi config, not sure I need it. If I remove it, django doesn't spawn threads and my WSGI won't spawns threads and I'll be using single threads for all requests? – eugene Jan 13 '16 at 12:15
  • 1
    Check the uWSGI documentation, I think you'll be using *separate subprocesses* instead. Which means that the connections *definitely* won't be shared across processes. Within each subprocess connections will be reused. – Martijn Pieters Jan 13 '16 at 12:16

2 Answers2

0

Maybe you already reuse connections

Any requests that you make within a session will automatically reuse the appropriate connection!

via http://docs.python-requests.org/en/latest/user/advanced/#keep-alive

But if you need to be sure you can put some debug prints inside urllib3 requests/packages/urllib3/poolmanager.py

histrio
  • 1,217
  • 13
  • 24
0

You can try increasing your logging verbosity, then look out for logs that look like:

"Starting new HTTPS connection (1): some.url:port"

This is how to make global logging more verbose:

import logging
logging.basicConfig(level=logging.DEBUG, format="%(message)s")

If the connection is being reused, you will only see one of those messages for any given url. If it is not being reused, you will see a different one each time a connection is established to the same url.

smac89
  • 39,374
  • 15
  • 132
  • 179