0

I'm facing a strange issue in which I'm trying to replace all NaN values in a dataframe with values taken from another one (same length) that has the relevant values.

Here's a glimpse for the "target dataframe" in which I want to replace the values: data_with_null

![enter image description here

Here's the dataframe where I want to take data from: predicted_paticipant_groups

enter image description here

I've tried:

data_with_null.participant_groups.fillna(predicted_paticipant_groups.participant_groups, inplace=True)

but it just fills all values NaN values with the 1st one (Infra)

enter image description here

Is it because of the indexes of data_with_null are all zeros?

Ben
  • 421
  • 6
  • 19
  • As dfs are same length, have you tried merging them and running gillan on a single df? – CreekGeek Jun 04 '21 at 18:08
  • `.fillna()` just takes a single value and replaces all NaN values with that. You probably just want to overwrite the `participant_groups` like so, no need to use `fillna` here: `df['participant_groups'] = df.predicted_paticipant_groups`? – fsimonjetz Jun 04 '21 at 18:08
  • You can use `data_with_null["participant_groups"]=predicted_paticipant_groups["participant_groups"].values` – DYZ Jun 04 '21 at 18:08

1 Answers1

3

Reset the index and try again.

data_with_null.reset_index(drop=True, inplace=True)
Jurgen
  • 119
  • 4
  • BTW, I'm still getting a warning 'Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/core/series.py:4529: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame' – Ben Jun 04 '21 at 18:28
  • You can learn more about that warning here: https://stackoverflow.com/questions/20625582/how-to-deal-with-settingwithcopywarning-in-pandas – Jurgen Jun 04 '21 at 18:37
  • I've also tried this: ```data_with_null = pd.merge(data_with_null, predicted_paticipant_groups, on='participant_groups')``` but I ended up with empty dataframe – Ben Jun 04 '21 at 18:45