df.fillna() does not replace all NaN values

Question

I'm trying to replace NaN values in my dataframe using:

values = {'pitch_type': 'UN', 'px': -1, 'pz': -1, 'pitch_type_prev': -1,'px_prev': -1, 
          'pz_prev': -1}

df_sample.replace(np.nan, values)

It fills all but two values. The returned output is :

pitch_type       px        pz     pitch_type_prev     px_prev         pz_prev
   UN          -1.000   -1.000        -1                 NaN           -1.000
   FF           0.416    2.963        -1                 NaN           -1.000
   FF          -0.191    2.347        FF           (0.0712, 0.508]      2.963

Any fix for this?

```values``` here is not a dataframe and it doesn't contain any Nan. — Panda50, Apr 21 '21 at 21:41

score 1 · Answer 1 · answered Apr 21 '21 at 21:47

1

You can use fillna:

df_sample.fillna(values)

Output:

  pitch_type     px     pz pitch_type_prev          px_prev  pz_prev
0         UN -1.000 -1.000              -1               -1   -1.000
1         FF  0.416  2.963              -1               -1   -1.000
2         FF -0.191  2.347              FF  (0.0712, 0.508]    2.963

answered Apr 21 '21 at 21:47

perl

9,826
1
10
22

that gives me a 'fill value must be in categories' error – Iris Apr 22 '21 at 03:19
1

@Iris It looks like you're trying to fill NaNs with values that are not in the list of categories. You can add those values as categories (see e.g. https://stackoverflow.com/questions/53664948/pandas-fillna-throws-valueerror-fill-value-must-be-in-categories) and then run `fillna` – perl Apr 22 '21 at 21:47

df.fillna() does not replace all NaN values

1 Answers1