Using the Python requests library, how can I trust a server TLS certificate when this certificate mentions an issuer that I can't access (untrusted root)?
In other words, I'd like to trust the public key provided in the server certificate. I don't want to completely disable certificate validation, and ideally the solution would not prevent requests to validate other valid certificates.
Using curl and openssl, I can manage to get this result, but I am unable to reproduce this using the requests library, even using the verify parameter.
Using the URL https://untrusted-root.badssl.com, which exposes a certificate with an untrusted root:
# Certificate issuer is not trusted
$ curl https://untrusted-root.badssl.com
curl: (60) server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
...
# Download the server certificate locally to badsslcom.crt
$ openssl s_client -showcerts -servername untrusted-root.badssl.com -connect untrusted-root.badssl.com:443 </dev/null 2>/dev/null | openssl x509 -outform pem > badsslcom.crt
# Now, curl accepts the server certificate
$ curl --cacert badsslcom.crt https://untrusted-root.badssl.com
<!DOCTYPE html>
<html>
...
</html>
The following Python code raises an exception however:
import requests
import os
import logging
logging.basicConfig(level=logging.DEBUG)
url = "https://untrusted-root.badssl.com/"
cert = "badsslcom.crt"
if not os.path.exists(cert):
raise ValueError('Can not find cert')
r = requests.get(url, verify=cert)
The exception is
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): untrusted-root.badssl.com
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 560, in urlopen
body=body, headers=headers)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 787, in _validate_conn
conn.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 252, in connect
ssl_version=resolved_ssl_version)
File "/usr/lib/python3/dist-packages/urllib3/util/ssl_.py", line 305, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.5/ssl.py", line 377, in wrap_socket
_context=self)
File "/usr/lib/python3.5/ssl.py", line 752, in __init__
self.do_handshake()
File "/usr/lib/python3.5/ssl.py", line 988, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib/python3.5/ssl.py", line 633, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 376, in send
timeout=timeout
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 589, in urlopen
raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./test_ssl.py", line 13, in <module>
r = requests.get(url, verify=cert)
File "/usr/lib/python3/dist-packages/requests/api.py", line 67, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3/dist-packages/requests/api.py", line 53, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 480, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 588, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 447, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)
How can I mirror the curl behaviour in Python? Adding the certificate to the Ubuntu 18.04 system store and pointing the REQUESTS_CA_BUNDLE environment variable to the system store did not help either.
