1

This is an updated based on the generous feedback that I've received from the Stackoverflow community. I'm still having some problems and need additional help:

I am using an iPython notebook version 2.7 on a Windows 10 system to try to web scrape the tables found in Wikipedia's Billboard Top 100 for the years 1992 through 2015. Full disclosure, that this is a class assignment but I am not getting the right support on the school forums.

Here is the code that I am trying to use:

yearstext = {}
for year in range(1992, 2015, 1):
    url=requests.get("http://en.wikipedia.org/wiki/Billboard_Year-End_Hot_100_singles_of_"+str(year))
    soup = BeautifulSoup(url.text, "html.parser")
    yearstext[year] = soup

This works (thanks to your help). However, it only gives me 2014 data. I need it to loop through the years and give me a dictionary as it's keys the years (1992 through 2014) and as values corresponding to these keys the text of the page being fetched. Do I need an append statement? I keep trying to implement one and I haven't gotten it to work.

Any help would be greatly appreciated.

James
  • 37
  • 1
  • 6
  • That's a completely different issue. That means that `r[2].find("a")` isn't finding anything, so you can't call `.get("href")` on it. Please update your question with the updated code/error, also, add the main [python] tag. – Morgan Thrapp Sep 16 '15 at 19:16

1 Answers1

0

As you said, Get an if-else check on the cleaner function

cleaner = lambda r: [int(r[0].get_text()), r[1].get_text(), r[2].get_text(), r[2].find("a").get("href") if r[2].find("a") else "Couldn't Find A Link"]

The general syntax of the if-else is (Putting a simple if-then statement on one line)

value_when_true if condition else value_when_false
Community
  • 1
  • 1
cchahadoop
  • 175
  • 2
  • 7