My longstanding Python 2.7.17 environment got corrupted, and so I installed Python 2.7.18 using the macOS 64-bit installer downloadable from python.org. Now I get errors when loading HTTPS sites using urllib2 but not requests.
Here is the environment, running macOS Mojave:
Python 2.7.18 (v2.7.18:8d21aa21f2, Apr 19 2020, 20:48:48)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Works fine with requests:
>>> import requests
>>> url='https://github.com'
>>> requests.get(url)
<Response [200]>
Fails with urllib2:
>>> import urllib2
>>> urllib2.urlopen(url)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 429, in open
response = self._open(req, data)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 447, in _open
'_open', req)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1241, in https_open
context=self._context)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1198, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)>
This occurs in both the plain 2.7.18 I installed and in virtual environments created from it. The problem is bigger than just urllib2, as certain database modules fail, too, that do not use urllib2. Moving to Python3 is not currently an option.
I tried pip install --upgrade certifi but that did not help.
Per [1] monkey-patching fixes the problem, but that is not a viable solution because this is destined to be production code.
Per [1] explicitly setting a non-verifying SSL context sometimes works, but again, it is not viable, plus it fails in certain third-party database interface code.
The answer may be in PEPP [2] or PEPP 493 [3], but the sun will swallow the earth before I understand those.
[1] urllib and "SSL: CERTIFICATE_VERIFY_FAILED" Error
