We know that requests.Session keeps a single TCP connection alive for multiple HTTP requests/responses sent through it. But what happens when requests is patched by gevent.monkey?
Since we can concurrently launch multiple HTTP requests through a single session (which has not yet built any TCP connection with the target server), would these concurrently launched requests be delivered through multiple TCP connections? Then would the session always build a new TCP connection for any new request?
Then, with gevent and requests, what to do if I want to establish a limited number of TCP connections and keep reusing them concurrently? Does gevent.pool only limit the number of concurrently running greenlets, not the number of alive TCP connections?
With preliminary experiments on Windows, I found that gevent initializes new TCP connections if multiple requests are launched concurrently from a single session. An example test code is given below:
from gevent import monkey, pool
monkey.patch_all()
import gevent
import requests
s = requests.Session()
def launch_one_request(url='https://www.baidu.com'):
s.get(url)
def launch_serial_requests(count=3):
for i in range(count):
launch_one_request()
gevent.sleep(1)
p = pool.Pool(5)
tasks = [p.spawn(launch_serial_requests) for _ in range(10)]
gevent.joinall(tasks)
tasks = [p.spawn(launch_serial_requests) for _ in range(10)]
gevent.joinall(tasks)
With the code above, we can use Windows resource monitor to insepct that 5 TCP connections are established, corresponding to the pool size. No additional TCP connection is established during the 2nd run of tasks.
