6

I am trying to access a website from behind corporate firewall using below:- 

password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
password_mgr.add_password(None, url, username, password)
auth_handler = urllib2.HTTPBasicAuthHandler(password_mgr)
opener = urllib2.build_opener(auth_handler) 
urllib2.install_opener(opener) 
conn = urllib2.urlopen('http://python.org')

Getting error 

URLError: <urlopen error [Errno 11004] getaddrinfo failed>

I have tried with different handlers (tried ProxyHandler also in slightly different way), but doesn't seem to work.

Any clues to what could be the reason for error and any different ways to supply the credentials and make it work?

Ayyappa Das
  • 191
  • 1
  • 2
  • 6
  • What kind of auth does the proxy want? If it demands `Proxy-Authenticate: NTLM` then it's impossible to make it work with Python, I'm sorry to say. – Colonel Panic Sep 16 '13 at 11:30

2 Answers2

5

If you are using Proxy and that proxy has Username and Password (which many corporate proxies have), you need to set the proxy handler with urllib2.

  proxy_url = 'http://' + proxy_user + ':' + proxy_password + '@' + proxy_ip
  proxy_support = urllib2.ProxyHandler({"http":proxy_url})
  opener = urllib2.build_opener(proxy_support,urllib2.HTTPHandler)
  urllib2.install_opener(opener)

HTTPBasicAuthHandler is used to provide credentials for the site which you are going to access and not for going through the proxy. The above snippet might help you.

Senthil Kumaran
  • 54,681
  • 14
  • 94
  • 131
  • I had tried this already, but it gave Http 401 Unauthorized as error. I suspect my corporate proxy is NTLM and the above methods might not suffice. – Ayyappa Das Jan 31 '11 at 05:17
  • Please try with http://code.google.com/p/python-ntlm/ Also try with the suggestion in this link, http://stackoverflow.com/questions/1481398/python-urllib2-https-and-proxy-ntlm-authentication/1793206#1793206 If both these work and plain urllib2 does not, then please file a report with bugs.python.org, if not already present. Guess, it needs to be worked upon. – Senthil Kumaran Jan 31 '11 at 05:25
  • python-ntlm didn't help - gave same getaddinfo error. Will try pycurl. Thanks a lot for the pointers. – Ayyappa Das Jan 31 '11 at 06:26
4

On Windows, I observed that python uses the IE Internet Options-> LAN Settings settings. So even if we use urllib2 to install opener and specify the proxy_url, it would continue to use the IE settings.

It worked fine finally, when I exported a system variable:

http_proxy=http://userid:pswd@proxyurl.com:port
j0k
  • 22,600
  • 28
  • 79
  • 90
Alec Chu
  • 41
  • 1
  • 1
    call me dumb but exported where, and how? Did you run this at the command-line? If so, I assume using `set`? – jamiet Mar 12 '17 at 22:54