pandas groupby ignoring NAs

Question

I have a DataFrame which has a lot of NAs. pandas's groupby operation is ignoring any combinations with NA in it. Is there a way to include NAs in groups? If not, what are the alternatives to pandas groupby? I really don't want to fill in NAs because the fact that something is missing is useful information.

Edit: I noticed that my question is exactly the same issue reported in groupby columns with NaN (missing) values Has there been any developments technology to get around this issue?

The question is a bit vague. Could you please show what you tried and what is your desired output? — abudis, Aug 07 '15 at 11:48

score 1 · Answer 1 · answered Aug 07 '15 at 07:55

1

I will use some kind of non-NA representation for NA only for groupby, which can't be confused with proper data (e.g. -999999 or 'missing')

df.fillna(-999999).groupby(...)

As inplace argument has default value False your original dataframe will not be affected.

answered Aug 07 '15 at 07:55

zuku

649
8
24

Thanks for the suggestion. Will df.fillna(-999999) work if my df has mixed data types such as string, float, datetime, etc.? – user5199987 Aug 07 '15 at 12:50
I've just tested it and it shouldn't be a problem for grouped columns. – zuku Aug 07 '15 at 13:34

pandas groupby ignoring NAs

1 Answers1

Linked