I'm pulling stock data from TD Ameritrade API and I want to store it in a DataFrame.
From the API I get a nested JSON object and when I put it in a data frame I get 4 columns: Index, Candles, Empty, Symbol. However inside of candles is a dictionary that I want as separate columns in the dataframe ('open','close',...)
I've tried json_normalize
and pd.io.json.json_normalize
neither gave me the desired result
import pandas as pd
import requests
from pandas.io.json import json_normalize
endpoint = r'https://api.tdameritrade.com/v1/marketdata/{}/pricehistory'.format('GOOG')
client_id = 'AMSAFI1234567'
payload = {'apikey':client_id,
'periodType': 'day',
'frequencyType': 'minute',
'frequency' :'1',
'period':'2',
'endDate': '1556158524000',
'startDate': '1554535854000',
'needExtendedHoursData':'true'}
content = requests.get(url = endpoint, params = payload)
data = content.json()
print(data)
Output:
{'candles': [{'open': 1260.25, 'high': 1260.5, 'low': 1260.0, 'close': 1260.28,
'volume': 2544, 'datetime': 1556029980000}, {'open': 1260.39, 'high': 1260.61,
'low': 1260.3501, 'close': 1260.3501, 'volume': 1703, 'datetime':
1556030040000}, {'open': 1260.35, 'high': 1260.59, 'low': 1260.07, 'close':
1260.56, 'volume': 2156, 'datetime': 1556030100000}, {'open': 1260.56, 'high':
1260.56, 'low': 1259.27, 'close': 1259.7, 'volume': 1320, 'datetime':
1556030160000}, {'open': 1260.06, 'high': 1260.06, 'low': 1259.56, 'close':
1259.56, 'volume': 800, 'datetime': 1556030220000},
....
'close': 1264.61, 'volume': 100, 'datetime': 1556146920000}, {'open': 1265.87,
'high': 1266.0, 'low': 1265.87, 'close': 1266.0, 'volume': 232, 'datetime':
1556147220000}], 'symbol': 'GOOG', 'empty': False}
Input:
pd.DataFrame(data)
Output:
Data frame with 4 columns ('Index', 'Candles', 'Empty', 'Symbol'). The Candles column is a dictionary. I'm trying to get all the keys in the dictionaries as columns and the key values as rows in the dataframe
