I have having trouble accessing HTTPS sites using urllib2. Here's what I've got:
import urllib2
import ssl
proxy = urllib2.ProxyHandler({'https':'http://username:password@proxy:port',
'http': 'http://username:password@proxy:port'})
opener = urllib2.build_opener(urllib2.HTTPHandler(),
urllib2.HTTPSHandler(),
proxy)
urllib2.install(opener)
url_secure = 'https://www.google.com'
url_nonsecure = 'http://www.google.com'
response = urllib2.urlopen(url_nonsecure)
d = response.read()
response.close()
print d
The above runs without issue. However, if i try to run the above using
response = urllib2.urlopen(url_secure)
I get
Traceback (most recent call last):
File "google_maps.py", line 25, in <module>
response = urllib2.urlopen(url_secure)
File "/Users/username/anaconda2/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/Users/username/anaconda2/lib/python2.7/urllib2.py", line 429, in open
response = self._open(req, data)
File "/Users/username/anaconda2/lib/python2.7/urllib2.py", line 447, in _open
'_open', req)
File "/Users/username/anaconda2/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/Users/username/anaconda2/lib/python2.7/urllib2.py", line 1241, in https_open
context=self._context)
File "/Users/username/anaconda2/lib/python2.7/urllib2.py", line 1198, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>
From this and this it seemed like the problem could be solved by creating a new context. So I tried both:
ctx=ssl._create_unverified_context()
and
ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
and ran
response = urllib2.urlopen(url_secure, context = ctx)
d = response.read()
response.close()
print d
which both threw the following error:
Traceback (most recent call last):
File "google_maps.py", line 25, in <module>
response = urllib2.urlopen(url_secure, context=ctx)
File "/Users/username/anaconda2/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/Users/username/anaconda2/lib/python2.7/urllib2.py", line 429, in open
response = self._open(req, data)
File "/Users/username/anaconda2/lib/python2.7/urllib2.py", line 447, in _open
'_open', req)
File "/Users/username/anaconda2/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/Users/username/anaconda2/lib/python2.7/urllib2.py", line 1241, in https_open
context=self._context)
File "/Users/username/anaconda2/lib/python2.7/urllib2.py", line 1198, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [Errno 8] nodename nor servname provided, or not known>
How can I access https sites?
