Web Scraping Error - Python

Question

My code is like this:

import urllib
import red

urlist=["asx:egs","aapl"]

i=0
while i<len(urlist):
  url = "https://au.finance.yahoo.com/quote/"+urlist[i]+"?p="+urlist[i]
  htmlfile = urllib.urlopen(url)
  htmlread = html.read()
  regex = "<span class='Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)'data-reactid='36'>(.+?)</span>"
  pattern =re.compile(regex)
  finalltext = re.findall(pattern,htmltext)
  print finalltext
  i+=1

though i get an error saying:

Traceback (most recent call last):
  File "python", line 9, in <module>
IOError: [Errno socket error] [Errno -2] Name or service not known

I tried searching the web though got no help. All help if appreciated.
------------------------------------------EDIT----------------------------------
I ran a debug:

import urllib
import re

urlist=["aapl","egs.ax"]

i=0
while i<len(urlist):
  url = "https://au.finance.yahoo.com/quote/"+urlist[i]+"?p="+urlist[i]
  print url
  htmlfile = urllib.urlopen(url)
  htmlread = html.read()
  regex = "<span class='Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)'data-reactid='36'>(.+?)</span>"
  pattern =re.compile(regex)
  finalltext = re.findall(pattern,htmltext)
  print finalltext
  i+=1

The result was:

https://au.finance.yahoo.com/quote/egs.ax?p=egs.ax

Traceback (most recent call last):
  File "python", line 10, in <module>
IOError: [Errno socket error] [Errno -2] Name or service not known

Still getting the same error even though the site is reachable.

For web scraping, I would strongly recommend to use adapted librairies. Simply using requests and beautifulsoup4 would be a massive step-up. Because [regex-based XML parsing is a bad idea](https://stackoverflow.com/q/8577060/4255615). — tleb, Jul 22 '17 at 08:27
Also `htmlread = html.read()` this line should be `htmlread = htmlfile.read()` — Bijoy, Jul 22 '17 at 08:27
I ran your code, its working, but output is an emplty list, also check line 14 — Bijoy, Jul 22 '17 at 08:31

Web Scraping Error - Python

0 Answers0