0

I am behind a proxy settings with the environment variables on the proxy settings set properly. The environment variables are working properly when doing pip installs for instance. I retrieve the proxy settings with the getproxies method, which I checked and returns the correct dict.

I am trying the following approach: import requests import urllib

r = requests.get('http://www.nu.nl', proxies=urllib.request.getproxies())

The error message I get:

ProxyError: HTTPSConnectionPool(host='www.nu.nl', port=443): Max retries 
exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.', 
OSError('Tunnel connection failed: 407 authenticationrequired',)))

What am I missing here?

EddyG
  • 675
  • 4
  • 13

2 Answers2

1

Ok, figured it out. The issue with my environment variables was that the username and password were not specified because I work in a single sign on environment. Therefore the password needs to be properly set. Just substituting my credentials in the proxy definition did not work, so I had to use the urllib opener to fix my issue. Now it works like a charm.

import urllib

username = 'userID'  # ex. ID
password = "password"  # password

targetUrl = "http://www.example.org/"

proxies = {
   'https':  'https://{}:{}@proxyAdress:port'.format(username, password)}
proxy = urllib.request.ProxyHandler(proxies)
opener = urllib.request.build_opener(proxy)
urllib.request.install_opener(opener)

with urllib.request.urlopen(targetUrl) as url:
    text = str(url.read())
EddyG
  • 675
  • 4
  • 13
0

The HTTP 407 code states that you need to authenticate to the proxy server.
The http response header Proxy-Authenticate will tell you what kind of authentication is required - print out the response headers.

It might also be a problem with the url...try removing the www bit

For reference : https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/407

bmat
  • 224
  • 3
  • 5
  • If I follow that link I still have no idea on what to do. Could you explain a bit more on what needs to be done? Thanks – EddyG Mar 05 '19 at 15:05
  • r.headers = Out[6]: {'Date': 'Tue, 05 Mar 2019 14:59:26 GMT', 'Content-Type': 'text/html', 'Cache-Control': 'no-cache', 'Content-Length': '3637', 'Proxy-Connection': 'Keep-Alive', 'Proxy-Authenticate': 'Negotiate, NTLM, Basic realm="McAfee Web Gateway"'} – EddyG Mar 05 '19 at 15:07
  • Is this print out what you meant? – EddyG Mar 05 '19 at 15:07
  • Yes... by the way does it work with this url : [https://nu.nl](https://nu.nl) ? note use of `http(s)` and removal of `www` – bmat Mar 05 '19 at 15:11
  • You might need to add your windows name and password see similar problem : https://stackoverflow.com/questions/34079/how-to-specify-an-authenticated-proxy-for-a-python-http-connection – bmat Mar 05 '19 at 15:16