ValueError: arrays must all be same length - Parse the JSON into Pandas DataFrame

Question

import requests
import json
import pandas as pd

data = requests.get("https://earthquake.usgs.gov/fdsnws/event/1/query?format=geojson")

json_data = data.json()

with open(r"C:\path\file.json",'w') as outfile:
    json.dump(json_data, outfile)

df = pd.read_json(r"C:\path\file.json")

when I tried to parse the json data into Pandas Dataframe, I get the below error: ValueError: arrays must all be same length

Can anyone help me out in this?

Traceback (most recent call last):
  File "c:/path/file.py", line 29, in <module>
    df = pd.read_json(r"C:\path\file.json")
  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\util\_decorators.py", line 199, in wrapper
    return func(*args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\util\_decorators.py", line 296, in wrapper
    return func(*args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\io\json\_json.py", line 618, in read_json
    result = json_reader.read()
  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\io\json\_json.py", line 755, in read
    obj = self._get_object_parser(self.data)
  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\io\json\_json.py", line 777, in _get_object_parser
    obj = FrameParser(json, **kwargs).parse()
  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\io\json\_json.py", line 886, in parse
    self._parse_no_numpy()
  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\io\json\_json.py", line 1118, in _parse_no_numpy
    self.obj = DataFrame(
  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\frame.py", line 468, in __init__
    mgr = init_dict(data, index, columns, dtype=dtype)
  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\internals\construction.py", line 283, in init_dict
    return arrays_to_mgr(arrays, data_names, index, columns, dtype=dtype)
  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\internals\construction.py", line 78, in arrays_to_mgr
    index = extract_index(arrays)
  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\internals\construction.py", line 397, in extract_index
    raise ValueError("arrays must all be same length")
ValueError: arrays must all be same length

In the JSON file you're working with, the dictionary values in the form of arrays for different keys must essentially be of the same length. Else add NaN to missing values — Raghul Raj, Dec 31 '20 at 06:20
Does this answer your question? [How to read a json-dictionary type file with pandas?](https://stackoverflow.com/questions/28373282/how-to-read-a-json-dictionary-type-file-with-pandas) — Raghul Raj, Dec 31 '20 at 06:25

score 0 · Accepted Answer · answered Dec 31 '20 at 16:27

Simpler approach is not to save to a file and use json_normalize()

import requests
import json
import pandas as pd

data = requests.get("https://earthquake.usgs.gov/fdsnws/event/1/query?format=geojson")
json_data = data.json()

pd.json_normalize(json_data["features"])

ValueError: arrays must all be same length - Parse the JSON into Pandas DataFrame

1 Answers1