1

I have a dictionary which contains a large no of data frames within it. I want to extract all the data frames from it and stores those values into a single dataframe. The dictionary is as shown:

Dictionary

One of the many dataframes which are stored in the dictionary is as shown:

Dataframe

I have tried using the 'pd.DataFrame.from_dict' method as shown:

wwwwww = pd.DataFrame.from_dict(res, orient='index')

but the output which is presented to me is garbled or is deprecated as shown:

Garbled output

Hence I did a further search and found the most trivial method of converting the dictionary to dataframe using pandas and used it as shown:

data2 = pd.DataFrame(res, index =[0])

Using this I got an value error as shown:

ValueError: Shape of passed values is (14, 501), indices imply (1, 501)

Hence I searched further and found a stack overflow answer as below:

ValueError: Shape of passed values is (1, 6), indices imply (6, 6)

In this the dictionary is passed as rows as shown below:

data2 = pd.DataFrame([res], index =[0])

Using this again I am getting a garbled value as shown:

corrupted data

How do I achieve my task of extracting all the dataframes from the dictionary into a single dataframe? Is there any efficient method for the same?

If the code is needed I attach is as shown:

import pandas as pd
import mysql.connector 
from nsepy import get_history
from datetime import date
stock =['3MINDIA','ABB','POWERINDIA','ACC','AIAENG'] //There are many values but I am including lesser values
res = dict(zip(stock,stock))
start = date (2019, 12, 1)
end = date (2020, 12, 10)

for stock_name in stock:
     data = get_history(symbol=stock_name['close'], start=start, end=end)
     res[stock_name]=data

data2 = pd.DataFrame([res], index =[0])


wwwwww = pd.DataFrame.from_dict(res, orient='index')
Huzefa Sadikot
  • 561
  • 1
  • 7
  • 22

1 Answers1

2

Since the values in the dictionary are already dataframes, I don't think you want to try to convert the dictionary into a DataFrame directly. Rather, iterate over the dictionary and add all the values to a new DataFrame.

Do you care about the keys in the dictionary?

If not one possible solution would be to make a list just using the values in the dictionary, then combine into a DataFrame with pd.concat.

lst = list(res.values())
df = pd.concat(lst)

If you do care about the keys in the dictionary, you could try adding the key as a column to each DataFrame.

for key, df in res.items():
    # create a column called "key name"
    df['key_name'] = key

lst = list(res.values())
df = pd.concat(lst)
Dustin Michels
  • 2,951
  • 2
  • 19
  • 31