How do I get the HTML of a web page if I have the url in python

Question

So let's say I have this URL: https://www.python.org/ and I want to download the page's source into a .txt file named python_source.txt

how would I do that?

See http://stackoverflow.com/questions/15138614/read-url-in-python and read up on `import urllib` — Brian Tiffin, Apr 29 '14 at 20:48
There are many ways to read a url's content in Python. There's no harm in putting them all here. — , Apr 29 '14 at 20:50
Stack Overflow is not a free code design and writing service. You need to show some effort into doing the work yourself before you can expect any assistance from us. As it stands your question is likely to be voted down and closed as not showing enough research on your part. — AdrianHHH, Apr 29 '14 at 21:54

score 2 · Accepted Answer · answered Apr 29 '14 at 20:48

2

Use urllib2, Here's how it's done:

response = urllib2.urlopen(url)
content = response.read()

Now you can save the content in any text file.

answered Apr 29 '14 at 20:48

score 0 · Answer 2 · answered Apr 29 '14 at 20:50

0

The python package urllib does just this. The documentation gives a very clear example on what you want to do.

import urllib.request
local_filename, headers = urllib.request.urlretrieve('http://python.org/')
html = open(local_filename)

answered Apr 29 '14 at 20:50

Ben Longo

2 Answers2