urllib2.proxyhandler in python 2.5

Question

In windows XP, python 2.5 and 2.6 I tested the following code:

import urllib2
proxy= urllib2.ProxyHandler({'http': '127.0.0.1:8080'})
opener = urllib2.build_opener(proxy)
urllib2.install_opener(opener)
urllib2.urlopen('http://www.google.com/')

In the above code I get a BadStatusLine exception from line 349 of httplib.py.

I have a proxy running at 127.0.0.1:8080 which works (I can set a browser to use it with proxyswitchy, and when it's on I can get to sites which are blocked when it's off [in China]).

if I change it to a socks proxy,

proxy= urllib2.ProxyHandler({'socks': '127.0.0.1:8080'})

Then the proxy is not used at all.

I got the code from the question at Proxy with urllib2 and it's almost exactly the same - what could be going wrong?

Update: urllib2 doesn't support socks proxies.

Eventually got it working with curl:

c = pycurl.Curl()

#stupid GFW
if settings.CHINA:
    c.setopt(pycurl.PROXY, '127.0.0.1')
    c.setopt(pycurl.PROXYPORT, 8087)
    c.setopt(pycurl.PROXYTYPE, pycurl.PROXYTYPE_SOCKS5)

Answer 1

The urllib2 ProxyHandler is not designed to support the SOCKS protocol. Perhaps this answer would help.

Answer 2

Assuming your local proxy is an HTTP proxy and not a socks proxy. Try this:

import urllib2
proxy= urllib2.ProxyHandler({'http': 'http://127.0.0.1:8080/'})
opener = urllib2.build_opener(proxy)
urllib2.install_opener(opener)
urllib2.urlopen('http://www.google.com/')

Answer 3

UPDATE: I am located behind the great firewall of china. This was compounding the problem. The gfw was both breaking connections and doing DNS poisoning.

I have not managed to get any of the urllib2 solutions working. But pycurl does seem to work and it gets around the "connection reset" problem. fb/twitter were still blocked though.

Adding their IPS to my hosts file works - so for a larger scale solution, setting up a dns proxy is necessary.