Python urllib2 > HTTP Proxy > HTTPS request

Question

This work fine:

import urllib2

opener = urllib2.build_opener(
                urllib2.HTTPHandler(),
                urllib2.HTTPSHandler(),
                urllib2.ProxyHandler({'http': 'http://user:pass@proxy:3128'}))
urllib2.install_opener(opener)
print urllib2.urlopen('http://www.google.com').read()

But, if http change to https:

...
print urllib2.urlopen('https://www.google.com').read()

There are errors:

Traceback (most recent call last):
  File "D:\Temp\6\tmp.py", line 13, in <module>
    print urllib2.urlopen('https://www.google.com').read()
  File "C:\Python26\lib\urllib2.py", line 124, in urlopen
    return _opener.open(url, data, timeout)
  File "C:\Python26\lib\urllib2.py", line 389, in open
    response = self._open(req, data)
  File "C:\Python26\lib\urllib2.py", line 407, in _open
    '_open', req)
  File "C:\Python26\lib\urllib2.py", line 367, in _call_chain
    result = func(*args)
  File "C:\Python26\lib\urllib2.py", line 1154, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "C:\Python26\lib\urllib2.py", line 1121, in do_open
    raise URLError(err)
URLError: <urlopen error [Errno 10060]

Why and how solve this problem?

if you are reading this: please mark the correct answer as correct. This is to ensure that nobody wastes their time trying to answer an already answered question. It is also a nice way of saying thanks to the person who came up with a solution for your problem — Sheena, Feb 11 '13 at 07:08

score 17 · Answer 1 · edited Jul 14 '10 at 22:01

17

Change this line:

urllib2.ProxyHandler({'http': 'http://user:pass@proxy:3128'}))

to this:

urllib2.ProxyHandler({'https': 'http://user:pass@proxy:3128'}))

It works fine for me.

edited Jul 14 '10 at 22:01

sth

222,467
53
283
367

answered Jul 14 '10 at 17:43

Fabiano

171
3

urllib2.ProxyHandler({'https': 'https://user:pass@proxy:3128'})) Change the second http to https if you want to use both http and an http proxy in your urllib2 – k9b Mar 01 '16 at 08:07

Oleg · Answer 2 · 2013-10-21T23:58:57.627

The documentation for urllib2 says the following:

Note: Currently urllib2 does not support fetching of https locations through a proxy. However, this can be enabled by extending urllib2 as shown in this recipe.

I must admit above recipe didn't work right away for Jython 2.5.3, but I'm still trying.

UPDATE: I applied this patch to Jython 2.5.3, and it worked for me. I can fetch HTTPS resources over a proxy server now.

UPDATE2: Here is the code to query HTTPS resources with Basic authentication over HTTP Proxy (DON'T FORGET TO INSTALL PATCH FIRST (see previous update)):

from suds.client import Client
from suds.transport.https import HttpAuthenticated

credentials = dict(username='...', password='...', proxy={'https': 'host:port', 'http': 'host:port'})
t = HttpAuthenticated(**credentials)
url = 'https://example.com/service?wsdl'
client = Client(url, transport=t)
print client.service.getFoo()

score 1 · Answer 3 · answered May 28 '10 at 09:32

On Windows, errno 10060 is a winsock error meaning the connection timed out. Are you able to reach https://www.google.com from the same machine using a web browser with a proxy set to http://user:pass@proxy:3128 ? Are you sure your proxy server can handle both https and http on the same port?

Python urllib2 > HTTP Proxy > HTTPS request

3 Answers3

Linked