1

I'm using the BeautifulSoup module to parse an html file that I want to extract certain information from. Specifically game scores and team names.

However, when I use the findAll function, it continually returns empty for a string that is certainly within the html. If someone can explain what I am doing wrong it will be greatly appreciated. See code below.

import urllib
import bs4
import re
from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup

my_url = 'http://www.foxsports.com/mlb/scores?season=2017&date=2017-05-09'
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()
# html parser
page_soup = soup(page_html, "html.parser")
container = page_soup.findAll("div",{"class":"wisbb_teams"})
print(len(container))
Nevermore
  • 7,141
  • 5
  • 42
  • 64
Jay Son
  • 11
  • 2

1 Answers1

2

I think the syntax your using is the old version of BeautifulSoup, try instead something like find_all snake_case (see the docs)

from bs4 import BeautifulSoup
# ...
page_html = uClient.read()
page_soup = BeautifulSoup(page_html, "html.parser")
list_of_divs = page_soup.find_all("div", class_="wisbb_name")
print(len(list_of_divs))

The older API used CamelCase, but bs4 uses snake_case

Also, notice that find_all takes can take a class_ parameter to find by class.

See this answer, https://stackoverflow.com/a/38471317/4443226, for some more info

Also, make sure you're looking for the correct classname! I don't see the class you're looking for, but rather these:

enter image description here

Community
  • 1
  • 1
Nevermore
  • 7,141
  • 5
  • 42
  • 64
  • Hey thank you! Can i Ask how you were able to find the potential classes containg wisbb? Also, at least when I inspect the element on the page, such a class exists. Do you know why this would be the case? The information I want is nested within many classes within the html, is that potentially why I cannot find it? – Jay Son May 12 '17 at 18:40
  • I got that image just from using the firefox inspection console, and searching for that string wisbb :) I don't see any wisb_teams – Nevermore May 12 '17 at 18:42