2

I just want to inquire regarding reusing the same connection while having a loop sending the same POST request. Assume I have this code:

import requests
import time
r = requests.Session()
url = "http://somenumbers.php"
while True:
    x = r.post(url)
    time.sleep(10)

Now according to the documentation of requests library

Excellent news — thanks to urllib3, keep-alive is 100% automatic within a session! Any requests that you make within a session will automatically reuse the appropriate connection! Note that connections are only released back to the pool for reuse once all body data has been read; be sure to either set stream to False or read the content property of the Response object

Does this work for the code up above? I am trying to prevent sending the same request in case the server freezes or a read timeout occurs. In Issue with sending POST requests using the library requests I go over the whole problem, and one of the suggestions is to reuse the connection, but

  • Isn't sending the same request on the same connection will just mean multiple entries, or is it going to fix the issue since it will only pull back when one entry is sent as the documentation states?

  • Assuming the latter is true, won't that affect performance and cause long delays since the request is trapped inside the connection?!

Community
  • 1
  • 1
Ahmed Al-haddad
  • 805
  • 2
  • 16
  • 41

1 Answers1

1

r.post is a blocking call. The function will only return once the request has been sent and a response is received. As long as you access x.content before the loop terminates, the next loop will re-use the underlying TCP connection.

Isn't sending the same request on the same connection will just mean multiple entries, or is it going to fix the issue since it will only pull back when one entry is sent as the documentation states?

requests doesn't cache the response. It will not check if a previous request having the same parameters was made. If you need that, you will have to build something on your own.

won't that affect performance and cause long delays since the request is trapped inside the connection

requests will only re-use an available connection. If no free connection exists, a new connection will be established. You can use requests.packages.urllib3.poolmanager.PoolManager to control the number of connections in the pool.

Paras
  • 642
  • 6
  • 16
  • Thank you so much sir. What I need is this kind of connection actually because my problem lies in sending multiple entries when the server lags and I get a read timeout, which causes me to retry sending again at the same problematic period. So if I don't use 'Session()' then it the connection will be non-blocking? How about if TCP fails in the case of a blocking connection, like when the server lags? Will TCP try to send the packet after the connection has been established but then a read timeout was encountered? – Ahmed Al-haddad Jul 13 '15 at 01:39
  • 1
    The connection will be blocking irrespective of using Session. The connection where will be closed once a timeout occurs. If there's no issue in the client, you should consider making your server non-blocking so that it can handle more connections, or scaling your server. – Paras Jul 13 '15 at 06:07
  • if the connection is blocking then does that mean that the packet that I send will be dropped once the connection closes? Because seemingly from my code, they are not dropped but rather sent, and since the server is freezed then all of them will be received at the same time(more than one because I try to send again in case it failed). – Ahmed Al-haddad Jul 13 '15 at 06:15
  • 2
    "Blocking" simply means that the task will not continue until the request completes or times out. It looks like you simply need to increase the timeout. – Paras Jul 13 '15 at 09:41