using requests with TLS doesn't give SNI support

Question

I'm using requests to communicate with a django app but

When I try

requests.get('https://mysite.com', verify=True)

I get the error:

hostname 'mysite.com' doesn't match either of '*.myhost.com', 'myhost.com'

However, when I look at the browser, or http://www.digicert.com/help/ the certificate looks fine and dandy.

My host suggested it was the lack of SNI support from requests (and Github seems to confirm that https://github.com/kennethreitz/requests/issues/749 ). Has anyone found a work-around using requests?

I edited Lukasa answer (the correct one) for a workaround to have SNI support in older requests in case you can't use the latest github version. Please ensure you have the required dependencies (not only the python ones, also openssl) installed in your system. It will be available once the edit is reviewed :) — AntonioMO, Sep 04 '13 at 01:25
I think it's better creating a new answer rather than perform a redical edit in the post of someone — Lucas Eduardo, Sep 04 '13 at 01:27
Well his answer is correct so I thought it wasn't nice to have another answer for another case, personally I really prefer one answer having all the info. But I'll post another answer then. — AntonioMO, Sep 04 '13 at 16:00
Curious, what operating system were you using to test this? Hitting SNI hosts works for me with Ubuntu Precise but not Lucid, and I cannot figure out why. http://stackoverflow.com/questions/24522757/why-does-ubuntu-12-04-precise-python-2-7-ssl-without-pyopenssl-work-against — Joe Shaw, Jul 02 '14 at 04:17

score 131 · Accepted Answer · edited Nov 05 '21 at 13:32

131

The current version of Requests should be just fine with SNI. Further down the GitHub issue you can see the requirements:

Try installing those packages and then give it another shot.

EDIT: As of Requests v2.12.1, ndg-httpsclient and pyasn1 are no longer required. The full list of required packages is now:

edited Nov 05 '21 at 13:32

baldr

2,891
11
43
61

answered Sep 02 '13 at 19:12

Lukasa

14,599
4
32
34

3

Thank you. I also saw this pull request https://github.com/kennethreitz/requests/pull/1347 which together with installing urllib3, ndg-httpsclient and pyasn1 solved it. – Massagran Sep 04 '13 at 15:59
The bug mentioned by @qarma is being tracked by [issue 1732](https://github.com/kennethreitz/requests/issues/1732) and is already fixed in master. – Lukasa Nov 25 '13 at 16:24
5

Is this fix still working? I get ``[Errno bad handshake] [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]``. – Roman Jun 27 '14 at 09:24
1

This fix absolutely is still working. Looks like the certificate isn't valid. – Lukasa Jun 27 '14 at 09:50
For Mac OSX Yosemite, you need pyOpenSSL==0.13. 0.14 didn't work for me. More inspiration at https://github.com/scrapy/scrapy/issues/922 – HostedMetrics.com Dec 07 '14 at 21:09
1

Just so you know, your answer is referenced in [this FAQ](http://docs.python-requests.org/en/latest/community/faq/) - at the end of the page. – Luís Cruz May 22 '15 at 21:50
Installing requests (2.18) also installs idna. So only pyopenssl package not installed as a dependency but required for SNI. – Paweł Bylica Sep 22 '17 at 10:52
Can you clarify what do you mean by "should be just fine with SNI"? Do you mean SNI is included by default and I do not need to specify any parameter for it? – user9371654 Mar 09 '19 at 18:13

score 22 · Answer 2 · edited Oct 01 '16 at 14:51

22

In order for me to get the accepted answer to work, I had to install a bunch of other packages, in this order:

yum install libffi-devel
yum install gcc
yum install openssl-devel
pip install urllib3
pip install pyopenssl
pip install ndg-httpsclient
pip install pyasn1

edited Oct 01 '16 at 14:51

Box Box Box Box

5,094
10
49
67

answered Oct 15 '14 at 22:00

Shane N

1,742
2
17
24

2

I also had to install: yum install python-devel before going to the pip installs.. – Kaos Jul 14 '15 at 09:02
Actually, for me, ndg-httpsclient installed all of those python packages except urllib3. Note to get a recent version of pip, though, or it won't work any how.. – Kaos Jul 14 '15 at 15:23
@ihue These installations were needed years ago to get around the fact that the Python SSL module in the standard library didn't support SNI. That's no longer the case, and using the newest versions of Python 2 or 3 don't require these extra installs. – dpoggi Jan 09 '17 at 19:36

score 12 · Answer 3 · answered Feb 03 '17 at 16:38

12

Install requests module like this. This will install the security package extras.

pip install requests[security]

answered Feb 03 '17 at 16:38

Nandeesh

2,683
2
30
42

Beware, this now needs setuptools-rust, and rust support. – progfan Mar 24 '21 at 00:32

score 5 · Answer 4 · answered Sep 04 '13 at 16:03

@Lukasa answer is correct with the present (from github) requests. Remember to add OpenSSL in your system too apart from the pip dependencies he mentions.

If for deployment reasons you prefer a stable requests version like the 1.2.3 in pip, you can monkey patch that one to work with SNI like this:

import requests


def fileno(self):
    return self.socket.fileno()


def close(self):
    return self.connection.shutdown()


requests.pyopenssl.WrappedSocket.close = close
requests.pyopenssl.WrappedSocket.fileno = fileno

score 5 · Answer 5 · answered Jul 30 '15 at 21:03

5

Or you can just use Python 2.7.9 and up:

"The entirety of Python 3.4's ssl module has been backported for Python 2.7.9. See PEP 466 for justification."

https://www.python.org/downloads/release/python-279/

answered Jul 30 '15 at 21:03

stantonk

1,922
1
18
24

9

I kept getting `requests.exceptions.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:590)` and I have `Python 2.7.10` – code-8 Nov 08 '16 at 15:54

score 2 · Answer 6 · answered Jul 22 '18 at 07:57

Copy my answer from Accessing https sites with IP address

On MAC High Sierra and Python 3.6.4, I tried the solution: requests toolbelt:HostHeaderSSLAdapter 1st, unfortunately, it doesn't work for me, then I tried forcediphttpsadapter, got it works finally.

The author explains everything in the readme part and has provided a sample script and it can be followed easily.

1.Install the library by pip install requests[security] forcediphttpsadapter

2.run the sample script:

import requests
from forcediphttpsadapter.adapters import ForcedIPHTTPSAdapter
session = requests.Session()
session.mount("https://example.com", ForcedIPHTTPSAdapter(dest_ip='1.2.3.4'))
response = session.get(
    '/some/path', headers={'Host': 'example.com'}, verify=False)

Note: For some cases, you may need to remove the prefix: 'www' from the url.

using requests with TLS doesn't give SNI support

6 Answers6

Linked