nhlscrapi - download Data error

Question

I am trying to get the statistics and game information of every game of an NHL Season. I am working with Stata. I found the package nhlscrapi and have written code to get all the data and statistics of a particular season:

# Import statements
# Notice how I import the whole modules and not the functions explicitly as given in the online example (good practice)
from nhlscrapi.games import game, cumstats
from nhlscrapi import constants
import csv


# Define season being considered:
season = 2012

# Get all stats they have defined
# Googled "get all methods of a class python" and found this:
# http://stackoverflow.com/questions/34439/finding-what-methods-an-object-has
# Also, needed to excclude some methods (ABCMeta, ...) after I checked what they do
# (I did that with: "help(cumstats.METHODNAME)") and saw that they did not contain stats
methods = [method for method in dir(cumstats) if callable(getattr(cumstats, method)) and 
                                                 method != 'ABCMeta' and 
                                                 method != 'AccumulateStats' and
                                                 method != 'ShotEventTallyBase' and
                                                 method != 'abstractmethod' and
                                                 method != 'TeamIncrementor' and
                                                 method != 'EF' and
                                                 method != 'St']

# Set up dictionary with all stats
cum_stats = {method: getattr(cumstats, method)() for method in methods}

print('All the stats:', cum_stats.keys())

# Now, look up how many games were in the regular season of the year 2012
maxgames = constants.GAME_CT_DICT[season]

# If one is interested in all the home coaches (as an example), one would first set up an empty list,
# and gradually fill it:
thingswewant_keys = ['home_coach', 'away_coach', 'home', 'away', 'attendance', 'Score', 'Fenwick']
thingswewant_values = {key: [] for key in thingswewant_keys if not key in cum_stats.keys()}
thingswewant_values.update({key+'_home': [] for key in cum_stats.keys()})
thingswewant_values.update({key+'_away': [] for key in cum_stats.keys()})

# Now, loop over all games in this season
for i in range(**12**):
    # Set up object which queries database
    # If one enters the following command in ipython: "help(game.Game)", one sees also alternative ways to set up
    # query other than the one given in the example
    ggames = game.Game(game.GameKey(season, game.GameType.Regular, i+1), cum_stats=cum_stats)

    # This object 'ggames' now contains all the information of 1 specific game.
    # To concatenate all the home coaches for example, one would do it like this

    for key in thingswewant_keys:
        if not key in cum_stats.keys():
            # First case: Information is attribute of ggames (e.g. home_coach)
            if not key in ['home', 'away', 'attendance']:
                thingswewant_values[key] += [getattr(ggames, key)]

            # Second case: Information is key of ggames.matchup (e.g. home)
            if key in ['home', 'away', 'attendance']:
                thingswewant_values[key] += [ggames.matchup[key]]

    # Third case: Information is a cum_stat
    # Figure out home_team and away team
    hometeam = ggames.matchup['home']
    awayteam = ggames.matchup['away']

    for key in cum_stats.keys():
        thingswewant_values[key+'_home'] += [ggames.cum_stats[key].total[hometeam]]
        thingswewant_values[key+'_away'] += [ggames.cum_stats[key].total[awayteam]]

# Make one single table out of all the columns
results = [tuple([key for key in thingswewant_values.keys()])]
results += zip(*[thingswewant_values[key] for key in thingswewant_values.keys()])

# Write to csv
with open('brrr.csv', 'wb') as f:
    writer = csv.writer(f)
    writer.writerows(results)

The problem now is that in every season, after a certain game, the code stops and spits out following error:

Traceback (most recent call last):
  File "C:/Users/Dennis/Downloads/AllStatsExcell.py", line 67, in <module>
    thingswewant_values[key+'_home'] += [ggames.cum_stats[key].total[hometeam]]
  File "C:\Python27\lib\site-packages\nhlscrapi\games\game.py", line 211, in cum_stats
    return self.play_by_play.compute_stats()
  File "C:\Python27\lib\site-packages\nhlscrapi\games\playbyplay.py", line 95, in compute_stats
    for play in self._rep_reader.parse_plays_stream():
  File "C:\Python27\lib\site-packages\nhlscrapi\scrapr\rtss.py", line 56, in parse_plays_stream
    p_obj = parser.build_play(p)
  File "C:\Python27\lib\site-packages\nhlscrapi\scrapr\rtss.py", line 130, in build_play
    p['vis_on_ice'] = self.__skaters(skater_tab[0][0]) if len(skater_tab) else { }
  File "C:\Python27\lib\site-packages\nhlscrapi\scrapr\rtss.py", line 159, in __skaters
    if pl[0].text.isdigit():
AttributeError: 'NoneType' object has no attribute 'isdigit'

In the 2012 season, this occurs after game 12. Therefore I just run for the game 12 in season 2012.

ggames1=game.Game(game.GameKey(2012, game.GameType.Regular, 12),cum_stats=cum_stats
ggames1.cum_stats['ShootOut'].total 

In ShootOut, for example, it crashes. But if I run this line again I get the results.

I don't know how to fix this.

If I just could get the csv file of all the games, even if there are some missing values I would be very happy.

Answer 1

First, you need to do some debugging yourself. The error explicitly states:

File "C:/Users/Dennis/Downloads/AllStatsExcell.py", line 67, in thingswewant_values[key+'_home'] += [ggames.cum_stats[key].total[hometeam]]

That means on line 67 of your program there is an error. At the bottom it shows you what that error is:

AttributeError: 'NoneType' object has no attribute 'isdigit'

This means that you are attempting to get the attribute isdigit on the value of an object that is NoneType. As you might surmise, NoneType objects don't have any contents.

This is the offending line, along with the preceding for block:

for key in cum_stats.keys():
    thingswewant_values[key+'_home'] += [ggames.cum_stats[key].total[hometeam]]

What you want to do is probably the following:

for key in cum_stats.keys():
    try:
        thingswewant_values[key+'_home'] += [ggames.cum_stats[key].total[hometeam]]
    except Exception as e:
        print(e)
        print("key={}".format(key)
        print("hometeam={}".format(hometeam)
        print("ggames.cumstats={}".format(s[key].total[hometeam])

This is a basic error catching block. The first print line should tell you the exception. The following ones inform you as to the state of various things you're utilizing in the offending line. Your job is to figure out which thing is NoneType (it may not be one of the ones I provided) and then, after that, figure out why it is NoneType. Essentially: look at the data you have and are trying to manipulate in that block. Something is missing in it.