0
  • Python 2.7.9
  • requests 2.17.3

We observe that when we call this from multiple threads (we run two to eight threads concurrently):

    with requests.Session() as s:
        s.put((apiserverUrl + str(id_num) + '/'), headers=headers,
                       auth=auth, data=jsonPayload)

That sometimes there is no client socket available, and it fails with a 10048 winsock error.

In some docs I reviewed, it appears that requests.Session() self closes, and in other places it indicates that it is held open for some default timeout.

What is the proper way to get the socket properly closed after each request?

Jonesome Reinstate Monica
  • 6,618
  • 11
  • 65
  • 112
  • The problem might be the number of sockets opened at a given time and not whether the socket was closed properly or not. – mad_ Aug 30 '18 at 18:32
  • @mad_ Fair point. We don't run very many threads, OP updated. I imagine the local port pool is much larger than our thread count, I am strongly suspecting that if we simply close our ports after use that we will be good to go. – Jonesome Reinstate Monica Aug 30 '18 at 18:34
  • You probably need to close the request each time. `s.put(...).close()` – Hitobat Aug 30 '18 at 19:31

1 Answers1

3

Requests library relies on urllib3 for an underlying HTTPConnection

And by default, urllib3 keeps the TCP connection alive.

There are a few ways to specify otherwise. I prefer including the header 'Connection':'close':

with requests.Session() as s:
    s.put((apiserverUrl + str(id_num) + '/'), headers={'Connection':'close'},
                   auth=auth, data=jsonPayload)

(more options on this answer: Python-Requests close http connection)

dbJones
  • 762
  • 1
  • 10
  • 31