11

I'm using Requests to connect to a RESTful API. The server I would like to reach use ssl with self-singed certificate.

cafile = "gateway.pem"
r = requests.get(request, auth=('admin', 'password'), verify=cafile)

the problem is I'm getting SSLError of hostname mismatch. there should be a way to disable the hostname checking without disabling certificate validation, as in many java implementations, but I can't find how to do it with requests in python.

stacktrace:

Traceback (most recent call last):
  File "<pyshell#43>", line 1, in <module>
    r = requests.get(request, auth=("admin", "password"), verify='gateway.pem')
  File "C:\Python27\lib\site-packages\requests-2.0.0-py2.7.egg\requests\api.py", line 55, in get
    return request('get', url, **kwargs)
  File "C:\Python27\lib\site-packages\requests-2.0.0-py2.7.egg\requests\api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python27\lib\site-packages\requests-2.0.0-py2.7.egg\requests\sessions.py", line 357, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python27\lib\site-packages\requests-2.0.0-py2.7.egg\requests\sessions.py", line 460, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python27\lib\site-packages\requests-2.0.0-py2.7.egg\requests\adapters.py", line 358, in send
    raise SSLError(e)
SSLError: hostname '10.76.92.70' doesn't match u'lital.com'

How can this be done?

tshepang
  • 12,111
  • 21
  • 91
  • 136
user3480498
  • 111
  • 1
  • 1
  • 4

6 Answers6

8

Requests doesn't allow this directly, however you can provide a custom transport adapter which uses the features of the underlying urllib3. The usage of transport adapters is covered in the requests documentation.

This code is not tested, but should work.

from requests.adapters import HTTPAdapter
from requests.packages.urllib3.poolmanager import PoolManager


# Never check any hostnames
class HostNameIgnoringAdapter(HTTPAdapter):
    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = PoolManager(num_pools=connections,
                                       maxsize=maxsize,
                                       block=block,
                                       assert_hostname=False)


# Check a custom hostname
class CustomHostNameCheckingAdapter(HTTPAdapter):
    def cert_verify(self, conn, url, verify, cert):
        #      implement me
        host = custom_function_mapping_url_to_hostname(url)
        conn.assert_hostname = host
        return super(CustomHostNameCheckingAdapter,
                     self).cert_verify(conn, url, verify, cert)

In detail the assert_hostname param works as follows: If None use the hostname from the URL, if False suppress hostname checking, if a custom string validate against this string.

t-8ch
  • 2,583
  • 14
  • 18
  • Thanks for that. I actually tried this as you can see below, but found out that after changing to use my costume adapter, no certificate varification is performed at all (instead of only hostname verification). I passed requests a corrupted pem file and got valid response. – user3480498 Apr 03 '14 at 08:05
  • 1
    class MyAdapter(requests.adapters.HTTPAdapter): def init_poolmanager(self, connections, maxsize, block): self._pool_connections = connections self._pool_maxsize = maxsize self._pool_block = block self.poolmanager = requests.packages.urllib3.PoolManager(num_pools=connections, maxsize=maxsize, block=block, assert_hostname=False) – user3480498 Apr 03 '14 at 08:05
  • 2
    I'm then doing this: s = requests.Session() s.mount('https://', MyAdapter()) r = s.get(request, auth=("admin", "admin"), verify=cafile) – user3480498 Apr 03 '14 at 08:10
  • Any idea what is worng here and why this disables the certificate validation? – user3480498 Apr 03 '14 at 08:10
  • If I use a corrupted pem file openssl blows up for me `requests.exceptions.SSLError: [Errno 0] _ssl.c:343: error:00000000:lib(0):func(0):reason(0)` – t-8ch Apr 03 '14 at 21:29
7

I'm a little late to the party but requests_toolbelt looks like it might help if you install version 0.7.0 or newer (my ubuntu 16.04 only has 0.6.0): https://toolbelt.readthedocs.io/en/latest/adapters.html#hostheaderssladapter

From the link:

Example usage:
>>> s.mount('https://', HostHeaderSSLAdapter())
>>> s.get("https://93.184.216.34", headers={"Host": "example.org"})
lane
  • 633
  • 12
  • 18
3

Did you look into the SSLContext.check_hostname parameter? You should be able to set it to False, and it should not check the hostname:

context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
context.verify_mode = ssl.CERT_REQUIRED
context.check_hostname = True
context.load_default_certs()

The only limitation is that this only works in Python 3.4 and later.

Reference: https://docs.python.org/3/library/ssl.html#ssl.SSLContext.check_hostname

nobody
  • 19,814
  • 17
  • 56
  • 77
Alecz
  • 1,951
  • 1
  • 19
  • 18
-3

It looks like this feature has been added to the latest version of requests. I've verified and it's working for me - just pass verify=False on the request - as shown in the example in the documentation:

requests.get('https://api.github.com', verify=False)
mojoken
  • 1,256
  • 1
  • 15
  • 27
  • The problem with this approach is that it turns off certificate validation completely, it doesn't just turn off the hostname check. In the question, the desired check is for "the presented certificate is signed by the certificate given in cafile", which means that certificate verification needs to remain enabled at the requests level. – ncoghlan Mar 17 '21 at 07:17
  • Does not works, the verify argument can not disable hostname validation. – e-info128 May 15 '21 at 21:53
-4

http://docs.python-requests.org/en/latest/user/advanced/#ssl-cert-verification

verify keyword is a flag, not for providing certfile. You provided there non-empy string, which resolves to True in boolean context.

Use cert= keyword to provide path to certificate files, or disable verification with verify=False.

EDIT: although documentation says you can actually pass CA path to verify=, but there are no examples. It would be helpful to see whole traceback you are receiving.

m.wasowski
  • 6,329
  • 1
  • 23
  • 30
  • When I'm doing the same call with verify=True I'm getting different error of "certificate verify failed". So looks like it is possible to pass the pem file to verify=. when I'm doing the call with cert=cafile instead of verify=cafile, I'm getting SSLError: [Errno 336265225] _ssl.c:351: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib. I added the traceback to the message. – user3480498 Mar 31 '14 at 10:56
  • what is your request? Looks like CA is hooked up properly, but it is signed for 'lital.com' and you connect by IP. Hence errors. – m.wasowski Mar 31 '14 at 11:00
  • right, the hostname in the certificate ('lital.com') isn't really the server's hostname. That's the reason I want to disable the hostname matching/checking. In java this can be done with ALLOW_ALL_HOSTNAME_VERIFIER. I couldn't find something similar in requests. – user3480498 Mar 31 '14 at 11:31
  • 2
    I don't want to disable entirely the certificate validation, only the hostname checking. – user3480498 Mar 31 '14 at 12:09
  • OK, now I get it. This is optional client feature as [docs say](rfc6125) and it may be unsupported in _request_, especially since it's security hazard. You can `try:` and if you catch this particular error, do manual check and if all seems ok for you, re-request with verify=False. Nothing else comes to my mind. – m.wasowski Mar 31 '14 at 12:23
  • PS. it doesn't look like it is supported: https://github.com/kennethreitz/requests/blob/master/requests/adapters.py but this may help you: http://stackoverflow.com/questions/18061640/ignore-certificate-validation-with-urllib3 – m.wasowski Mar 31 '14 at 12:25
-5

If this is only for testing only then just add an entry in /etc/hosts to your local system (assuming you have access).

user3644848
  • 1