ValueError: arrays must all be same length, json link

Question

import pandas as pd

url = 'https://site.web.api.espn.com/apis/common/v3/sports/basketball/nba/statistics/byathlete?region=us&lang=en&contentorigin=espn&isqualified=true&page=2&limit=50&sort=offensive.avgPoints%3Adesc'
df = pd.read_json(url)

I've been trying to export this link into JSON, but I'm getting this error:

 ValueError: arrays must all be same length

Data set can be seen after clicking on this link.

Answer 1

The main issue is, that JSON does not have fully filled every item, so every item can have different length.

You can solve the problem by extracting only those features you are interested in. For example like this:

# imports
import pandas as pd
import requests

# constants
URL = 'https://site.web.api.espn.com/apis/common/v3/sports/basketball/nba/statistics/byathlete?region=us&lang=en&contentorigin=espn&isqualified=true&page=2&limit=50&sort=offensive.avgPoints%3Adesc'

# save data to JSON
r = requests.get(URL)
data = r.json()

# extract only some features
df = pd.DataFrame(
    [
        {
            'id': item['athlete']['id'],
            'uid': item['athlete']['uid'],
            'firstName': item['athlete']['firstName'],
            'lastName': item['athlete']['lastName'],
        } 
        for item in data['athletes']
    ]
)

# print result
print(df.head())
        id                  uid firstName lastName
0  3133601  s:40~l:46~a:3133601  Devonte'   Graham
1  3112335  s:40~l:46~a:3112335    Nikola    Jokic
2  3133628  s:40~l:46~a:3133628     Myles   Turner
3  3917376  s:40~l:46~a:3917376    Jaylen    Brown
4  4277811  s:40~l:46~a:4277811    Collin   Sexton