System-level solution to Python SSL: CERTIFICATE_VERIFY_FAILED

Question

The IT security team at our business uses an intermediate certificate (ZScaler) to validate SSL traffic. This creates problems with any utility or API wrapper that uses SSL, e.g. httplib2, requests, etc.

I've found the solutions for when my code calls the modules, e.g. ssl_verify=False and so on, but the issue is when deep in the recesses of someone else's code these libs get called.

For example:

service = googleapiclient.discovery.build('vision', 'v1')

The error it throws:

---------------------------------------------------------------------------
SSLHandshakeError                         Traceback (most recent call last)
<ipython-input-4-70a8fbe53fc9> in <module>()
----> 1 service = googleapiclient.discovery.build('vision', 'v1')

/usr/local/anaconda/lib/python2.7/site-packages/oauth2client/_helpers.pyc in positional_wrapper(*args, **kwargs)
    131                 elif positional_parameters_enforcement == POSITIONAL_WARNING:
    132                     logger.warning(message)
--> 133             return wrapped(*args, **kwargs)
    134         return positional_wrapper
    135 

/usr/local/anaconda/lib/python2.7/site-packages/googleapiclient/discovery.pyc in build(serviceName, version, http, discoveryServiceUrl, developerKey, model, requestBuilder, credentials, cache_discovery, cache)
    226     try:
    227       content = _retrieve_discovery_doc(
--> 228         requested_url, discovery_http, cache_discovery, cache)
    229       return build_from_document(content, base=discovery_url, http=http,
    230           developerKey=developerKey, model=model, requestBuilder=requestBuilder,

/usr/local/anaconda/lib/python2.7/site-packages/googleapiclient/discovery.pyc in _retrieve_discovery_doc(url, http, cache_discovery, cache)
    273   logger.info('URL being requested: GET %s', actual_url)
    274 
--> 275   resp, content = http.request(actual_url)
    276 
    277   if resp.status >= 400:

/usr/local/anaconda/lib/python2.7/site-packages/httplib2/__init__.pyc in request(self, uri, method, body, headers, redirections, connection_type)
   1657                     content = ""
   1658                 else:
-> 1659                     (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
   1660         except Exception, e:
   1661             if self.force_exception_to_status_code:

/usr/local/anaconda/lib/python2.7/site-packages/httplib2/__init__.pyc in _request(self, conn, host, absolute_uri, request_uri, method, body, headers, redirections, cachekey)
   1397             auth.request(method, request_uri, headers, body)
   1398 
-> 1399         (response, content) = self._conn_request(conn, request_uri, method, body, headers)
   1400 
   1401         if auth:

/usr/local/anaconda/lib/python2.7/site-packages/httplib2/__init__.pyc in _conn_request(self, conn, request_uri, method, body, headers)
   1317             try:
   1318                 if hasattr(conn, 'sock') and conn.sock is None:
-> 1319                     conn.connect()
   1320                 conn.request(method, request_uri, body, headers)
   1321             except socket.timeout:

/usr/local/anaconda/lib/python2.7/site-packages/httplib2/__init__.pyc in connect(self)
   1090                 # something else (such as SSL protocol mismatch).
   1091                 if getattr(e, 'errno', None) == ssl.SSL_ERROR_SSL:
-> 1092                     raise SSLHandshakeError(e)
   1093                 else:
   1094                     raise

SSLHandshakeError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)

Is there a system-wide or global fix for this kind of issue?

Have you tried pointing to your company/zscaler certificates when you build your http object? check out this answer: https://stackoverflow.com/a/27856913 — MACanazon, Aug 09 '17 at 16:49

score 1 · Answer 1 · answered Aug 09 '17 at 19:47

Have you tried pointing to your company/zscaler certificates when you build your http object?

check out this answer: https://stackoverflow.com/a/27856913

It looks like oauth2client is the imported module that is causing the error.

You will need to pass the positional arg 'http' in the oauth2client function you are using.

For instance, if you are using oauth2client.tools.run_flow, you would need to build your http object and pass it as an arg to this function:

http = httplib2.Http(disable_ssl_certificate_validation=True)
tools.run_flow(flow, store, flags, http=http)

score 1 · Answer 2 · answered Aug 26 '22 at 21:09

The best system level solutions I can think of right now are to either:

A) Distribute the corporate certificate bundle(which includes the Zscaler CA certs) from a universal internal URL and use it to update system trust stores automatically when new infrastructure is provisioned, or

B) Use a standardized environment setup script that sets variables to specify certificate paths for all your commonly used tools.

Environment script example:

CORP_CA_BUNDLE="\path\to\IT\CA_bundle"

export SSL_CERT_FILE="${CORP_CA_BUNDLE}" #openssl
export REQUESTS_CA_BUNDLE="${CORP_CA_BUNDLE}" #requests
export AWS_CA_BUNDLE="${CORP_CA_BUNDLE}" #botocore
export CURL_CA_BUNDLE="${CORP_CA_BUNDLE}" #curl
export HTTPLIB2_CA_CERTS="${CORP_CA_BUNDLE}" #httplib2
#nothing for urllib3?

You can't always modify code to fix this problem and it is quite a headache keeping all the settings of various libraries and tools straight.

Thanks for collecting this list of env vars. That was usefule. A little correction imho: AWS_CA_BUNDLE is for boto3, not for botocore. Botocore uses urllib3 with certifi and therefore uses a hardcoded list. — udondan, Oct 21 '22 at 15:07

System-level solution to Python SSL: CERTIFICATE_VERIFY_FAILED

2 Answers2