42

I would retrieve some information from an ABB G13 gateway that offer a RESTful JSON API. API is hosted by the gateway via https endpoint. Basic authentication mechanism is used for authentication. However all traffic goes through SSL layers.

On linux with command:

curl -s -k -X GET -u user:password https://host/meters/a_serial/power

All goes well!

I'm trying to write a script for windows in Python 2.7.10 with Requests 2.8.1 and with this code:

import requests
requests.get('https://host/meters/a_serial/power', auth=('user', 'password'))

I have this error:

Traceback (most recent call last):
  File "C:/Users/mzilio/PycharmProjects/pwrgtw/test.py", line 20, in <module>
    requests.get('https://host/meters/a_serial/power', auth=('user', 'password'))
  File "C:\Python27\lib\site-packages\requests\api.py", line 69, in get
    return request('get', url, params=params, **kwargs)
  File "C:\Python27\lib\site-packages\requests\api.py", line 50, in request
    response = session.request(method=method, url=url, **kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python27\lib\site-packages\requests\adapters.py", line 433, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:590)

I've searched for a solution and I've tried to fix with this code:

import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.poolmanager import PoolManager
import ssl

class MyAdapter(HTTPAdapter):
    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = PoolManager(num_pools=connections,
                                       maxsize=maxsize,
                                       block=block,
                                       ssl_version=ssl.PROTOCOL_TLSv1)

s = requests.Session()
s.mount('https://', MyAdapter())
s.get('https://host/meters/a_serial/power')

But it doesn't work for me cause I get this error:

Traceback (most recent call last):
  File "C:/Users/mzilio/PycharmProjects/pwrgtw/test.py", line 16, in <module>
    s.get('https://host/meters/a_serial/power')
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 480, in get
    return self.request('GET', url, **kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python27\lib\site-packages\requests\adapters.py", line 433, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:590)

I'm stuck on this problem. Could someone help me? Thanks!

mzilio
  • 421
  • 1
  • 4
  • 4
  • Refer the stackoverflow question https://stackoverflow.com/questions/18578439/using-requests-with-tls-doesnt-give-sni-support/18579484#18579484 – mntl_ Oct 29 '15 at 09:47
  • 2
    With the fix refered, a new error: `requests.exceptions.SSLError: ("bad handshake: SysCallError(-1, 'Unexpected EOF')",)` – mzilio Oct 29 '15 at 09:52
  • 1
    Did you ever fix this? I am getting the same with an unrelated issue – A G Apr 08 '16 at 10:13
  • Can't this randomly occur if internet connection shuts down in the middle of the script ? – Overdrivr Oct 14 '20 at 14:35
  • 1
    Haunted me for a week, very sporadic. I had all the most up to date software and ssl stack. Problem was I had two conflicting DNS entries for the same domain, one without an A record. Deleted that one and had to wait TTL 48h. – Wolfgang Kuehn Nov 12 '20 at 15:27

9 Answers9

21

This thing worked for me, just make sure whether these modules are installed or not, if not then install them, following are:

pip install ndg-httpsclient

pip install pyopenssl

pip install pyasn1

It removed my SSLError: EOF occurred in violation of protocol (_ssl.c:590) error.

Hope it helps.

enigma
  • 1,029
  • 10
  • 11
14

I found it was going through a proxy when it should have connected to the server directly.

I fixed this by doing

unset https_proxy
A G
  • 997
  • 2
  • 18
  • 36
14

Step 1: Check that Python supports TLS 1.1

You may have a Python setup that only supports TLS 1.0 – not TLS 1.1 or above.

You can check it like this:

Python 3

from urllib.request import urlopen
urlopen('https://www.howsmyssl.com/a/check').read()

Python 2

from urllib2 import urlopen
urlopen('https://www.howsmyssl.com/a/check').read()

(If you get urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)> you may have to disable certificate verification. NOTE: doing this will disable SSL protections against evildoers who would impersonate or intercept traffic to that website - see https://en.wikipedia.org/wiki/Man-in-the-middle_attack )

import ssl
urlopen('https://www.howsmyssl.com/a/check', context=ssl._create_unverified_context()).read()

Check the output for the key tls_version. If it says TLS 1.0 and not TLS 1.1 or TLS 1.2 that could be the problem.

If you're using a virtualenv, be sure to run the command inside.

Step 2: Install Python with a newer version of OpenSSL

In order support TLS 1.1 or above, you may need to install a newer version of OpenSSL, and the install Python again afterwards. This should give you a Python that supports TLS 1.1.

The process depends on your operating system – here's a guide for OS X.

virtualenv users
For me, the Python outside of my virtualenv had TLS 1.2 support, so just I removed my old virtualenv, and created a new one with the same packages and then it worked. Easy peasy!

PaulMcG
  • 62,419
  • 16
  • 94
  • 130
qff
  • 5,524
  • 3
  • 37
  • 62
  • 1
    Tried above and urlopen showed tls_version as 1.2 but still I get the same error : ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:579). Any other ideas on how to fix this issue? – Rama Apr 03 '19 at 00:55
  • @Rama In that case you might be having a different issue. Like the missing `ndg-httpsclient` package as mentioned here https://stackoverflow.com/a/46467942/118608. I only know that the above answer (my answer) worked for me – qff Apr 03 '19 at 21:00
  • 2
    one-line check from the command line: `python -c "from urllib.request import urlopen ; print(urlopen('https://www.howsmyssl.com/a/check').read())"` – johntellsall Aug 14 '19 at 22:35
  • what version of openssl is required? – Harry Moreno Apr 03 '21 at 18:04
  • @HarryMoreno Check [this SO answer](https://stackoverflow.com/a/27545567/118608). TLS 1.1 is supported in Openssl 1.0.1 or later. – qff Apr 07 '21 at 08:06
  • What could be the cause of the above code giving me `urllib.error.URLError: ` ? I pass the exact string given above to `urlopen()`, and the error is the same for every well-formed URL. – Daniel Saner Jul 12 '22 at 14:52
5

I had exactly the same error, turns out that I didn't have ndg-httpsclient installed, see my original issue raised in github.

YSC
  • 38,212
  • 9
  • 96
  • 149
James Lin
  • 25,028
  • 36
  • 133
  • 233
3

If you are getting this error for intermediate requests, you can refer to the solution mentioned in https://github.com/requests/requests/issues/3391.

Basically, if you are making a lot of requests to a server and facing this issue with some of those requests you can use Session to just retry the requests.

Let me know if this works.

varun
  • 1,473
  • 1
  • 9
  • 15
  • 2
    My service is doing 100+ requests to same server (different parameters) but after 70-80 request was getting this error. Changed the implementation to use Session and so far no error. – Ricardo stands with Ukraine May 24 '22 at 11:09
2

The only time I have seen errors of this nature have been times that I was using requests to screen scrape data and I was using a multiprocessing library. I would either try your code without the pool manager or split your app into two apps, one that doles out urls and another that consumes them.

The client-server pair here should give you some ideas. I was able to scale my client out horizontally when I used the hacky server code to load all urls into a Queue in memory just before app.run. Hope that helps.

hughdbrown
  • 47,733
  • 20
  • 85
  • 108
2

Close you proxy and try it again.

Lane
  • 4,682
  • 1
  • 36
  • 20
0

ENV: Python 3.10, www.howsmyssl.com returns tls_version: TLS 1.3:

For poor guys like me who MUST make query through proxy, this cloud be blame on your incorrect HTTPS proxy setting (perhaps you aren't set it, but python somehow believes you've set it, don't know why exactly, maybe because you've set the http proxy), you need to set it "properly".

I'm using windows10, haven't set it before, after set it, python works normally, give it a try.

boholder
  • 100
  • 2
  • 6
0

try to run: pip install urllib3==1.25.11

It works for me anyway.

sdzz gndrc
  • 21