Pandas build dataframe from a list of single dicts

Question

I have a list of single dicts:

foo = [
{'A': 'a1'}, {'B': 'b1'},{'C': 'c1'},
 {'A': 'a2'}, {'B': 'b2'}, {'C': 'c2'},
 {'A': 'a3'}, {'B': 'b3'},{'C': 'c3'}
]

I want to build a DataFrame like this.

I tried this solution to group this list of dictionaries into a single dictionary Combine values of same keys in a list of dicts

bar = {
    k: [d.get(k) for d in foo]
    for k in set().union(*foo)
}

pd.DataFrame(bar)

But the output does not look good.

Can anyone help on this?

Answer 1

Transform to a dict of columns first:

from collections import defaultdict                                                                

bar = defaultdict(list)                                                                            

for dict_ in foo: 
    for key, value in dict_.items(): 
        bar[key].append(value) 

Then it becomes trivial:

pd.DataFrame(bar)                                                                                  
# Out 
#     A   B   C
# 0  a1  b1  c1
# 1  a2  b2  c2
# 2  a3  b3  c3

Answer 2

You could change your dictionary comprehension to filter missing values:

bar = {
    k: [d[k] for d in foo if k in d]
    for k in set().union(*foo)
}

Edit: The solution @Marat posted will work too.