Splitting a Dataframe of a Dict. to a new DataFrame with keys and values

Question

I have a dataframe of strings. The current dataframe looks like this: Current dataframe

Each datapoint contains a Dictionary like below:

"{'Index': 1, 'TimeSpent': 74088, 'RealInc': 'Obstacle_bef', 'IdentifiedIncident': 'Obstacle', 'TrLev': 7, 'TakeOverDecision': 'stay_put'},{'Index': 2, 'TimeSpent': 11336, 'RealInc': 'Obstacle_after_success', 'IdentifiedIncident': 'Pedestrian', 'TrLev': 7 },{'Index': 3, 'TimeSpent': 38594, 'RealInc': 'Cyclist_before', 'IdentifiedIncident': 'Cyclist', 'TrLev': 7, 'TakeOverDecision': 'stay_put'},{'Index': 4, 'TimeSpent': 16011, 'RealInc': 'Cyclist_after_success', 'IdentifiedIncident': 'Pedestrian', 'TrLev': 7 }".

I would like to make a new dataframe where each colomn represents the key of that dictionary. I have tried to use eval(), as well as using apply like this. But I think because every other dict is missing the key of 'TakeOverDecision', the apply does not work on it.

Any suggestions or guidance on how to split this to a dataset which make my dataset looks like below would be great! Desired dataframe

score 0 · Answer 1 · answered Sep 15 '19 at 15:58

0

Did you try the from_dict function

import pandas as pd
df = pd.from_dict(data)

answered Sep 15 '19 at 15:58

abhilb

5,639
2
20
26

if you mean "pd.DataFrame.from_dict(data)", yes I did, it basically returns what I currently have! – EPUVA Sep 15 '19 at 16:02

score 0 · Accepted Answer · answered Sep 15 '19 at 15:59

0

Maybe this could help.

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.from_dict.html

import pandas as pd
data = {'col_1': [3, 2, 1, 0], 'col_2': ['a', 'b', 'c', 'd']}
pd.DataFrame.from_dict(data)
   col_1 col_2
0      3     a
1      2     b
2      1     c
3      0     d

answered Sep 15 '19 at 15:59

aayush_malik

161
13

Thanks @AMal, from_dict works for one single dict object but in my case I have a tuple (it is the output of the eval function) with a dict as an element. do you have any solutions that I could convert the tuple to the dict and convert to the dataframe? – EPUVA Sep 15 '19 at 16:13
How about running a loop for all the elements? Performing the same operation on the entire tuple `for val in values` – aayush_malik Sep 15 '19 at 16:17

Splitting a Dataframe of a Dict. to a new DataFrame with keys and values

2 Answers2