Web-Scraping with Python on Canopy

Question

I'm having trouble with this line of code in which I want to print the 4 stock prices for the companies listed. My issue is that, while there are no errors when I run it, the code only prints out empty brackets where the stock prices should go. This is the source of my confusion.

import urllib2
import re

symbolslist = ["aapl","spy","goog","nflx"]
i = 0

while i<len(symbolslist):
    url = "http://money.cnn.com/quote/quote.html?symb=' +symbolslist[i] + '"
    htmlfile = urllib2.urlopen(url)
    htmltext = htmlfile.read()
    regex = '<span stream='+symbolslist[i]+' streamformat="ToHundredth" streamfeed="SunGard">(.+?)</span>'
    pattern = re.compile(regex)
    price = re.findall(pattern,htmltext)
    print "the price of", symbolslist[i], " is ", price
    i+=1

score 1 · Accepted Answer · edited May 23 '17 at 12:08

Because you don't pass the variable:

 url = "http://money.cnn.com/quote/quote.html?symb=' +symbolslist[i] + '"
                                                         ^^^^^
                                                      a string not the list element

Use str.format:

url = "http://money.cnn.com/quote/quote.html?symb={}".format(symbolslist[i])

Also you can iterate directly over the list, no need for a while loop, never parse html with a regex, use a html parse like bs4 and your regex is also wrong. There is no stream="aapl" etc.. what you want is the span where streamformat="ToHundredth" and streamfeed="SunGard";

import urllib2
from bs4 import BeautifulSoup

symbolslist = ["aapl","spy","goog","nflx"]


for symbol in symbolslist:
    url = "http://money.cnn.com/quote/quote.html?symb={}".format(symbol)
    htmlfile = urllib2.urlopen(url)
    soup = BeautifulSoup(htmlfile.read())
    price = soup.find("span",streamformat="ToHundredth", streamfeed="SunGard").text
    print "the price of {} is {}".format(symbol,price)

You can see if we run the code:

In [1]: import urllib2

In [2]: from bs4 import BeautifulSoup

In [3]: symbols_list = ["aapl", "spy", "goog", "nflx"]

In [4]: for symbol in symbols_list:
   ...:         url = "http://money.cnn.com/quote/quote.html?symb={}".format(symbol)
   ...:         htmlfile = urllib2.urlopen(url)
   ...:         soup = BeautifulSoup(htmlfile.read(), "html.parser")
   ...:         price = soup.find("span",streamformat="ToHundredth", streamfeed="SunGard").text
   ...:         print "the price of {} is {}".format(symbol,price)
   ...:     
the price of aapl is 115.57
the price of spy is 215.28
the price of goog is 771.76
the price of nflx is 97.34

We get what you want.

Now I'm getting this error after trying your code. It flags line 11 and says: AttributeError: 'NoneType' object has no attribute 'text' — Kainesplain, Sep 16 '16 at 15:27
The code output is contained in the answer, if you get different output using the symbols in the answer you are either using the code incorrectly or you are not getting the correct source for some reason. Telling me you get an attribute error without more context is a bit hard to debug — Padraic Cunningham, Sep 16 '16 at 16:14

Web-Scraping with Python on Canopy

1 Answers1