Python Proxy Error With Requests Library

Question

I am trying to access the web via a proxy server in Python. I am using the requests library and I am having an issue with authenticating my proxy as the proxy I am using requires a password.

proxyDict = { 
          'http'  : 'username:mypassword@77.75.105.165', 
          'https' : 'username:mypassword@77.75.105.165'
        }
r = requests.get("http://www.google.com", proxies=proxyDict)

I am getting the following error:

Traceback (most recent call last):
File "<pyshell#13>", line 1, in <module>
r = requests.get("http://www.google.com", proxies=proxyDict)
File "C:\Python27\lib\site-packages\requests\api.py", line 78, in get
:param url: URL for the new :class:`Request` object.
File "C:\Python27\lib\site-packages\requests\api.py", line 65, in request
"""Sends a POST request. Returns :class:`Response` object.
File "C:\Python27\lib\site-packages\requests\sessions.py", line 187, in request
def head(self, url, **kwargs):
File "C:\Python27\lib\site-packages\requests\models.py", line 407, in send
"""
File "C:\Python27\lib\site-packages\requests\packages\urllib3\poolmanager.py", line     127, in proxy_from_url
File "C:\Python27\lib\site-packages\requests\packages\urllib3\connectionpool.py", line    521, in connection_from_url
File "C:\Python27\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 497, in get_host
ValueError: invalid literal for int() with base 10: 'h6f2v6jh5dsxa@77.75.105.165'

How do I solve this?

Thanks in advance for your help.

It's good manners to mark correct answers as correct – Sheena Aug 26 '13 at 11:40 — Sheena, Aug 26 '13 at 11:40

score 17 · Answer 1 · edited Jan 25 '12 at 11:04

17

You should remove the embedded username and password from proxyDict, and use the auth parameter instead.

import requests
from requests.auth import HTTPProxyAuth

proxyDict = { 
          'http'  : '77.75.105.165', 
          'https' : '77.75.105.165'
        }
auth = HTTPProxyAuth('username', 'mypassword')

r = requests.get("http://www.google.com", proxies=proxyDict, auth=auth)

edited Jan 25 '12 at 11:04

guillermooo

7,915
15
55
58

answered Jan 14 '12 at 13:52

reclosedev

9,352
34
51

Thanks, I tried that and it solved the error but it still isn't connecting to the proxy. When I look at the returned headers (r.headers), one of them is 'proxy-connection': 'close', which seems to mean its taking too long to connect to the proxy, is there a way to authenticate before requesting any URLs? – Mark Collier Jan 14 '12 at 14:05
@MarkCollier, please try code from edited answer. It was error In first version. Added `HTTPProxyAuth`, now it should work. – reclosedev Jan 14 '12 at 14:11
from requests.auth import HTTPProxyAuth Traceback (most recent call last): File "", line 1, in from requests.auth import HTTPProxyAuth ImportError: cannot import name HTTPProxyAuth – Mark Collier Jan 14 '12 at 14:16
@MarkCollier, upgrade `requests`: `pip install --upgrade requests` – reclosedev Jan 14 '12 at 14:20
@MarkCollier, you mean proxy connection error, or cannot import name? I've tested this code on local proxy server and it works. When i'm entering wrong username or password, `r.status_code` is `407`. What is yours? – reclosedev Jan 14 '12 at 14:42
@MarkCollier, what is your `requests.__version__`? I have `0.9.1`. Maybe `pip install --upgrade requests` failed, or you are testing code in interactive shell, and it has started before upgrade? Try to launch code from file. – reclosedev Jan 14 '12 at 14:55
In case this helps, this worked for me, using `requests==1.1`. – Juan Carlos Coto Feb 11 '13 at 21:12
I have got the same error. I am with requests version 2.0.0. Any solution? – alextc Nov 25 '15 at 05:50
This fixed it for me - but does anyone know why? The answer isn't very thorough. – dmcmulle Jun 26 '17 at 23:47

score 3 · Answer 2 · answered Apr 24 '14 at 10:39

I've been having a similar problem on Windows and found the only way to get requests to work was to set the proxies as environment variables before I started Python. For you this would be something like this:

set HTTP_PROXY=http://77.75.105.165
set HTTPS_PROXY=https://77.75.105.165

You might also want to check is there's a specific port required, and if so set it after the url. For example, if the port is 8443 then do:

set HTTP_PROXY=http://77.75.105.165:8443
set HTTPS_PROXY=https://77.75.105.165:8443

see also https://stackoverflow.com/questions/5971312/how-to-set-environment-variables-in-python — raphael, Sep 19 '17 at 22:21

score 1 · Answer 3 · edited Sep 04 '19 at 00:54

1

You can use urllib library for this.

from urllib import request
request.urlopen("your URL", proxies=request.getproxies())

edited Sep 04 '19 at 00:54

Elliott

2,479
2
21
13

answered Apr 16 '14 at 07:15

Amit Sharma

131
1
2
12

Python Proxy Error With Requests Library

3 Answers3

Linked