Python Requests getting SSLerror

Question

Trying to make a simple get request using Requests session but I keep getting SSLerror for a specific site. I think maybe the problem is with the site (I did a scan using https://www.ssllabs.com, results are down bellow), but I cant be sure because I have no knowledge in this area :) I would sure like to understand what is going on.

A solution/explanation would be great, thanks!

The code:

import requests

requests.get('https://www.reporo.com/')

I'm getting the next error:

SSLError: [Errno bad handshake] [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]

---------------------------------------------------------------------------
SSLError                                  Traceback (most recent call last)
<ipython-input-7-cfc21b287fee> in <module>()
----> 1 requests.get('https://www.reporo.com/')

/usr/local/lib/python2.7/dist-packages/requests/api.pyc in get(url, **kwargs)
     63 
     64     kwargs.setdefault('allow_redirects', True)
---> 65     return request('get', url, **kwargs)
     66 
     67 

/usr/local/lib/python2.7/dist-packages/requests/api.pyc in request(method, url, **kwargs)
     47 
     48     session = sessions.Session()
---> 49     response = session.request(method=method, url=url, **kwargs)
     50     # By explicitly closing the session, we avoid leaving sockets open which
     51     # can trigger a ResourceWarning in some cases, and look like a memory leak

/usr/local/lib/python2.7/dist-packages/requests/sessions.pyc in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    459         }
    460         send_kwargs.update(settings)
--> 461         resp = self.send(prep, **send_kwargs)
    462 
    463         return resp

/usr/local/lib/python2.7/dist-packages/requests/sessions.pyc in send(self, request, **kwargs)
    571 
    572         # Send the request
--> 573         r = adapter.send(request, **kwargs)
    574 
    575         # Total elapsed time of the request (approximately)

/usr/local/lib/python2.7/dist-packages/requests/adapters.pyc in send(self, request, stream, timeout, verify, cert, proxies)
    429         except (_SSLError, _HTTPError) as e:
    430             if isinstance(e, _SSLError):
--> 431                 raise SSLError(e, request=request)
    432             elif isinstance(e, ReadTimeoutError):
    433                 raise ReadTimeout(e, request=request)

SSLError: [Errno bad handshake] [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]

I ran a scan at https://www.ssllabs.com and got the following:

SSL Report: reporo.com
Assessed on:  Sun Feb 22 21:42:57 PST 2015 | Clear cache Scan Another >>

    Server  Domain(s)   Test time   Grade
1   154.51.128.13 
Certificate not valid for domain name 
reporo.com
Sun Feb 22 21:40:53 PST 2015 
Duration: 9.167 sec -
2   198.12.15.168 
protected.ddosdefend.com 
Ready 
www.reporo.com

Sun Feb 22 21:41:02 PST 2015 
Duration: 115.189 sec   
F

The server is using an invalid certificate, just like the report says. Therefore, `requests` is refusing to connect, as the information transfer may not be secure. — MattDMo, Feb 23 '15 at 06:05

Steffen Ullrich · Accepted Answer · 2015-02-23T07:39:16.230

25

The certificate itself for www.reporo.com (not reporo.com) is valid, but it is missing a chain certificate as shown in the report by ssllabs:

Chain issues    Incomplete
....
2   Extra download  Thawte DV SSL CA 
Fingerprint: 3ca958f3e7d6837e1c1acf8b0f6a2e6d487d6762

The "Incomplete" and "Extra download" are the major points. Some browsers will have the missing chain certificate cached, others will do the download and other will fail. If you try the site with a fresh Firefox profile (which does not have any certificates cached) it will fail too.

You could download the missing chain certificates and use it as trusted CA certificate with the verify parameter for requests. Don't just disable validation because then you are open to man-in-the-middle attacks.

Step by step instruction:

Download the missing certificate at https://ssl-tools.net/certificates/vqgvhb-thawte-dv-ssl-ca (found by searching for the fingerprint given in the report from SSLLabs). Download the file in PEM format, i.e. https://ssl-tools.net/certificates/3ca958f3e7d6837e1c1acf8b0f6a2e6d487d6762.pem.
Download the root certificate at https://ssl-tools.net/certificates/91c6d6ee3e8ac86384e548c299295c756c817b81.pem (also found by searching for fingerprint).
Cat both files together into a new file chain.pem. Make sure that each of the files did end with a valid end of line character (which they do not as downloaded). The resulting file should look like this.

Modify your call to

requests.get('https://www.reporo.com/', verify = 'chain.pem')

edited Feb 23 '15 at 07:39

answered Feb 23 '15 at 06:14

Steffen Ullrich

114,247
10
131
172

2

Can you elaborate on how to download & use CA certificate? (or links on the subject) the subject is new to me and I would like to avoid messing up :) – Captain_Meow_Meow Feb 23 '15 at 06:34
3

I've added step-by-step instructions to the answer. – Steffen Ullrich Feb 23 '15 at 07:43
I've tried the solution to the letter as far as I see, but I keep getting SSLError: [Errno bad handshake] [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]. :/ – Captain_Meow_Meow Feb 23 '15 at 09:58
1

Looks like they've changed their site. Now it works with the default CA path instead, that is you don't need to add a specific path. The explicit path does not work any longer because the current certificate needs another trust chain. – Steffen Ullrich Feb 23 '15 at 13:09
Indeed it looks like their back to normal, how strange. But still thanks for the help, I hope now I could handle similar situations in the future if they'll arise :) – Captain_Meow_Meow Feb 25 '15 at 08:06
@SteffenUllrich Would you please elaborate a tad more on how you found the missing chain certificate, I admit I can't figure it out on own. Perhaps I'm doing something wrong...I donno – Albert Jan 19 '17 at 18:30
@Albert: just go over to [SSLLabs](https://www.ssllabs.com/ssltest/analyze.html) and let them analyze the site. They will show you if chain certificates are missing in the report. I've cited the relevant part from their report in my answer. – Steffen Ullrich Jan 19 '17 at 18:48
@SteffenUllrich I've run a test at SSLLabs and the report contains the following issue: `Chain issues Incomplete, Extra certs, Contains anchor` but it has no extra download part. Does it mean that in this case it's impossible to obtain the missing part of the cert? – Albert Jan 19 '17 at 19:00
@Albert: hard to tell without seeing the exact report. Could you please provide a link? – Steffen Ullrich Jan 19 '17 at 20:03
@SteffenUllrich sure, here it is: https://www.ssllabs.com/ssltest/analyze.html?d=spb.metro-cc.ru – Albert Jan 19 '17 at 20:22
1

@Albert: they've changed the interface since 2015. You now have to click "Click here to expand" to get to the exact details of the certificate path and which certificates are missing (i.e. extra download). You can find these certificates then for download by just search for the shown fingerprint using google or similar. – Steffen Ullrich Jan 19 '17 at 20:45
@SteffenUllrich well now things are getting relatively clear, sorry for one more silly question but which one is the root certificate? I googled a lot but now seem to be completely confused as to how to obtain that cert. Unless I'm not mistaken, in this case the cert's fingerprint is 02faf3e291435468607857694df5e45b68851868, the ones above are all the intermediate certs. Google found more than 7000 links but they don't seem right to me... – Albert Jan 21 '17 at 10:12
@Albert: the problem is not the root cert but the intermediate cert. Since this one is missing it can not build the trust chain to the local root cert. Therefore you would to add the missing intermediate as trusted and this one would be [104c63d2546b8021dd105e9fba5a8d78169f6b32](https://ssl-tools.net/subjects/0ba882a3e868b9b611daa8a0b510253d697b0245). – Steffen Ullrich Jan 21 '17 at 14:45
As an alternative to searching for the certs manually, you can use tools like https://whatsmychaincert.com to generate the chain (and tell requests to use it as trust store like described in the answer). – Stefan Apr 14 '22 at 10:57

score 17 · Answer 2 · answered Feb 23 '15 at 07:23

17

You can disable certificate verification:

requests.get('https://www.reporo.com/', verify=False)

but without certificate verification there is no man-in-the-middle attack protection.

answered Feb 23 '15 at 07:23

Colin

221
1
2

29

Please never ever suggest to disable validation without pointing out the security implications. – Steffen Ullrich Feb 23 '15 at 07:36

score 2 · Answer 3 · answered Nov 23 '17 at 23:11

2

Ran into similar issue and fixed by following:

pip install -U requests[security]

answered Nov 23 '17 at 23:11

Deqing

14,098
15
84
131

1

Stackoverflow says not the leave comments like "thanks" or "+1" but your simple instruction ended my search and deserves more visibility. – silian-rail Jul 18 '23 at 21:04

score 1 · Answer 4 · answered Jun 12 '17 at 22:50

1

I had the same error. Downgrading requests from requests-2.17.3 to requests-2.11.0 solved it for me.

pip uninstall requests
pip install requests==2.11.0

answered Jun 12 '17 at 22:50

Aleksei Denisov

27
2

1

Any idea what changes between those two versions would account for broken handshake? – jxramos Apr 23 '19 at 10:19

score 1 · Answer 5 · answered Jan 02 '21 at 02:05

Steffen Ullrich has the best answer for when it is the case that a website has valid certs as determined by the browser but the complete certification chain is not supplied by the server.

Sometimes just the root or intermediate cert is missing and browsers are equipped to download or verify the missing cert on the fly, but with the requests module you will have verify it manually. I used SSL labs to determine the full certificate chain as a .pem file. Enter the url to the website of interest, and wait for the test to complete. Then navigate to and expand "certification paths". There may be multiple trusted paths (or none, in which case the eventual request will not succeed), and off to the right you will see a download chain button.

Copy the full chain as text and save it as a .pem file. Then you pass the path of this .pem file to the requests function when you make a request:

r = requests.get(url, verify= "path/to/chain.pem")

score 0 · Answer 6 · answered Apr 09 '22 at 16:46

This error also occur when using MimT.

You can trust your CA by using requests.get('https://github.com', verify='/path/to/certfile'), or set its path to REQUESTS_CA_BUNDLE environment variable.

Ref: https://docs.python-requests.org/en/master/user/advanced/#ssl-cert-verification

Python Requests getting SSLerror

6 Answers6

Linked

Related