7

I do HTTP requests on Python 3.7, but if I have some errors during the execution and the script stops, all new HTTP requests (even after restart the script) have the errors: [Errno 10054] An existing connection was forcibly closed by the remote host.

I have to disable/enable my network, to remove the error. It's probably due to my PC/OS because the script works on RaspberryPi but not on my Windows 10. But I don't know how to fix it.

Here the minimum code to generate the error: 

import requests
import json
import urllib.request
import socket

    if __name__ == '__main__':

        params = json.dumps({"toto": "ABCD"}).encode('utf-8')

        try:
            head = {'content-type': 'application/json'}
            #replace http by https, to generate the error, re-writte http, and it will never work again
            url = 'http://www.google.com'
            with requests.post(url, data=params, headers=head) as response:
                print("All is OK: " + str(response))
        except (urllib.error.URLError, socket.timeout) as e:
            print("Error time out: " + str(e.args))
        except Exception as e:
            print("Uknown error: " + str(e.args))

Once I have the error, even pip.exe return the same error if I want to install a new module.

edit1:I tried something else:

I have another script which does only SQL requests, it works perfectly.

But once I have the error even this one has the problem: "Lost connection to MySQL server during query (%s)" % (e,)) pymysql.err.OperationalError: (2013, 'Lost connection to MySQL server during query ([WinError 10054] An existing connection was forcibly closed by the remote host)')

edit2: I installed bash on Ubuntu on Windows, and I do the same thing on the same PC but "OS different", I get ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

I tried on other PC (Windows 7), same problem as mine.

@djvg I checked openSSL version (1.1.0i) and TSL version (1.2), so it looks compatible with the link you give. And this is the output of pip freeze (thanks for the tips, I didn't know btw):

certifi==2018.11.29 chardet==3.0.4 cycler==0.10.0 idna==2.8 kiwisolver==1.0.1 matplotlib==3.0.2 numpy==1.15.4 PyMySQL==0.9.3 pyparsing==2.3.0 python-dateutil==2.7.5 requests==2.21.0 six==1.12.0 urllib3==1.24.1

@M.Spiller I don't use a "special network", if I test on my professional network, it doesn't work but with another error (probably proxy issue), so I use another network with no particular security. But I don't think is a security issue, because this error appears once the script stops. And the script works well on raspberry pi. Could be an OS issue ? Like the network board doesn't close the connection correctly the first time. But I don't know how to verify that...

@UserX I can't disable the firewall: company politic. I'm agree with you, it's probably the cause. Is it possible to open another connection ? Or to force the closing even when the script crash ?

P. Paul-Alexandre
  • 159
  • 1
  • 11
  • Unfortunately it works for me (Ubuntu Bionic) so I'm afraid can't help you, but just as a sidenote for the beautiful requests library - from 2.4.2 you can use `requests.post(url, json={"toto": "ABCD"})` and it will take care of everything else. – mfrackowiak Jan 11 '19 at 10:53
  • Perhaps raw sockets? – 0xInfection Jan 11 '19 at 10:57
  • I couln't reproduce your issue, what version of requests library do you use? Maybe 'Connection' argument in headers will help: ```head = {'content-type': 'application/json', 'Connection':'close'}``` – VadimK Jan 17 '19 at 06:19
  • Please clarify what you meant by `even after restart the script`... – Dima Tisnek Jan 17 '19 at 08:47
  • If you could paste the output from `pip freeze`, that may help reproduce the issue. Did you look into your TLS version, as suggested e.g. [here](https://stackoverflow.com/a/50746109) and [here](https://stackoverflow.com/a/38502727)? Also [this patch](https://github.com/django/django/commit/934acf1126995f6e6ccba5947ec8f7561633c27f) for a similar issue with Django's test server may be interesting (and [this](https://code.djangoproject.com/ticket/29849#comment:4)). – djvg Jan 17 '19 at 09:03
  • @DimaTisnek I meant: the script is not running anymore and I relaunch it (I don't know how to explain more than that) – P. Paul-Alexandre Jan 18 '19 at 12:55
  • I suggest responding to the rest of the questions in the comments, preferably [edit]ing your question with all available information. I added a bounty to your question because it's well-written, has a proper MCVE and seems to be reproducible, but you'll have to help others solve the problem as much as you can, otherwise the additional visibility the bounty gives you is wasted. – Andras Deak -- Слава Україні Jan 18 '19 at 13:25
  • Is there something "special" in your network, like for example a proxy server you should use? – MSpiller Jan 18 '19 at 13:36
  • 1
    Did you try disabling firewall? – HariUserX Jan 18 '19 at 17:54
  • This does seem like egress filtering, or programme/executable-based filtering. +1 to @UserX – Dima Tisnek Jan 21 '19 at 02:50
  • re: `don't know how to verify that` -- use something simple that hammers a URL, for example `ab` command from Apache. See if repeated access to same/similar URL gets blocked. – Dima Tisnek Jan 21 '19 at 02:54
  • Yet another option is that your OS leaves dead connections in bad state and your MySQL **server** runs out of connections. Check the server with e.g. `netstat` or mysql own status command in a clean state vs. when the error has happened. – Dima Tisnek Jan 21 '19 at 02:56
  • check out code 10054 in https://learn.microsoft.com/en-us/windows/desktop/WinSock/windows-sockets-error-codes-2. Most likely its a firewall or ssl certificate issue. – HariUserX Jan 21 '19 at 07:22
  • Your question behind with _"if I have some errors during the execution"_. Can you clarify that? What do you have to do to start having errors? Do you have to make many requests? Do requests have to be sent to google? It sounds like you may be blocked by Google for breaking their terms of service (using scripts on Google's http servers this way is not allowed iirc). – zvone Jan 21 '19 at 08:34
  • @zvone as I written in the code example, to generate an error I change http to httpS. But The error could be other thing. – P. Paul-Alexandre Jan 21 '19 at 09:42
  • 1
    Tried with *Python* ***2.7***, ***3.5***, ***3.6***, ***3.7*** on *Win 10* (at least 5 times each), and every time I got "*All is OK: *". – CristiFati Jan 22 '19 at 20:06

1 Answers1

0

I answer to my own question:

for my case it's a firewall issue. I can't modify its parameters due to company politic but with a computer with no firewall it works. It's not a SW solution and it's a expensive workaround but it's better than nothing.

P. Paul-Alexandre
  • 159
  • 1
  • 11