Python urllib urlretrieve behind proxy

Question

I looked into the documentation of urllib but all I could find on proxies was related to urlopen. However, I want to download a PDF from a given URL and store it locally but using a certain proxy server. My approach so far which did not work:

import urllib2

proxies = {'http': 'http://123.96.220.2:81'}
opener = urllib2.FancyURLopener(proxies)
download = opener.urlretrieve(URL, file_name)

The error is AttributeError: FancyURLopener instance has no attribute 'urlretrieve'

Serial · Accepted Answer · 2013-07-03T14:52:23.927

4

I beleive you can do something like this:

import urllib2

proxy = urllib2.ProxyHandler({'http': '123.96.220.2:81'})
opener = urllib2.build_opener(proxy)
urllib2.install_opener(opener)

with open('filename','wb') as f:
    f.write(urllib2.urlopen(URL).read())
    f.close()

since urllib2 doesnt have urlretrieve you can just use urlopen to get the same effect

you must have got the docs confused becuase urllib2 also doesnt have FancyURLopener thats why youre getting the error

urllib2 is much better when handling proxies and such

for more info look here Urllib2 Docs

edited Jul 03 '13 at 14:52

answered Jul 03 '13 at 14:09

Serial

7,925
13
52
71

I just checked `urllib2.urlopen(URL, filename)`. This does not work as it does not download and store the PDF given the filename. – orschiro Jul 03 '13 at 14:31
all the gz file that I downloaded using this are corrupted, they are not when I use google chrome – Heetola Oct 10 '19 at 12:38

score 1 · Answer 2 · edited May 23 '17 at 11:59

1

You've got the right idea you're just missing a few things.

proxy = urllib2.ProxyHandler({'http': '127.0.0.1'})
opener = urllib2.build_opener(proxy)
urllib2.install_opener(opener)
urllib2.urlopen('http://www.google.com')

edited May 23 '17 at 11:59

Community

1
1

answered Jul 03 '13 at 13:59

John

13,197
7
51
101

Python urllib urlretrieve behind proxy

2 Answers2

Linked