Python : append/merge dicts to pandas in a pythonic way

Question

I'm new to python, I'm trying to append result from my iteration loop that give out different number of keys each time in the loop .. something like

 result i=1 => {key1:a, key2:b}
 result i=2 => {key2:c, key3:d , key4:z}

I want my result to be a dataframe something like

key1  key2    key3  key4
a       b    None  None
None    c      d     z

what is the most efficient way to do something

for i in range(10):
    do something 
    result dict

combine

score 1 · Accepted Answer · answered Oct 10 '13 at 12:34

You can either append each dict to your DataFrame as you loop:

for i in range(10):
    # create dict
    df.append( created_dict )

or create a list of the created dicts and pass this into the DataFrame constructor

df = pd.DataFrame(my_list_of_dicts)

However, if you have a lot of dicts then appending to the list will become expensive and the first code snippet would be better.

score 1 · Answer 2 · edited May 23 '17 at 11:57

Something like this might work:

In [1]: import pandas as pd

In [2]: dic1 = {'key1':'a', 'key2':'b'}       

In [3]: dic2 = {'key2':'c', 'key3':'d' , 'key4':'z'}

In [4]: keys = sorted(dic1.viewkeys() | dic2.viewkeys())

In [5]: pd.DataFrame([[d.get(k) for k in keys]
                                         for d in (dic1, dic2)], columns=keys)                        
Out[5]: 
   key1 key2  key3  key4
0     a    b  None  None
1  None    c     d     z

Instead of using just sorted I'd suggest you to use natural sort for sorting such keys.

Python : append/merge dicts to pandas in a pythonic way

2 Answers2