Dealing with missing values when converting a list of dictionaries into a dataframe

Question

Is there a way to deal with missing values when converting a list of dictionaries into a pandas dataframe? Sometimes the dictionary entries in different orders so I have to deal with each column separately.

Here is an example:

p = [
 {'c': 53.13,'n': 1,'t': 1575050400000},
 {'t': 1575048600000,'c': 53.11}
  ]

And here is what I have been trying:

import pandas as pd
df = pd.DataFrame([{
        "c": t["c"],
        "n": t["n"],
        "t": t['t']}
        for t in p])

I get a KeyError: 'n' because the entry for 'n' is missing in the second dictionary. Is there a way of handling that to just put a NaN when the entry is missing?

perl · Accepted Answer · 2021-06-26T13:36:10.737

1

You can instantiate a DataFrame from p as an argument:

df = pd.DataFrame(p)
df

Output:

       c    n              t
0  53.13  1.0  1575050400000
1  53.11  NaN  1575048600000

edited Jun 26 '21 at 13:36

answered Jun 26 '21 at 13:28

perl

9,826
1
10
22

1

sorry i deleted my comment because it was obvious from your output. – helloimgeorgia Jun 26 '21 at 13:31
@helloimgeorgia Yes, makes sense :) deleted mine as well then to avoid confusion – perl Jun 26 '21 at 13:31
@helloimgeorgia In fact, you don't even need `from_dict` here; just using `pd.DataFrame(p)` would do the same thing; updated my answer – perl Jun 26 '21 at 13:38

Dealing with missing values when converting a list of dictionaries into a dataframe

1 Answers1