Why does replacing dataframe columns with dictionary just create nan's?

Question

I want to replace the column 'GeneID' with my dictionary.

This creates just nan's.. Does someone know why this happens?

 df
Out[107]: 
        Region     GeneID  DistanceValue
0           BG   79677107            0.0
       ...        ...            ...
1097355  CERus       1415            NaN
[1097360 rows x 3 columns]

replacing it with final_dictionary1

df["GeneID"] = df["GeneID"].map(final_dictionary1)

whereas final_dictionary1 looks like:

...
 '52856': 'Mtg2',
 '19886': 'Ros1',
 '16008': 'Igfbp2',
 '14747': 'Cmklr1',
 '13401': 'Dmwd',
 '12545': 'Cdc7',
 '28113': 'Tinf2',
 '71833': 'Dcaf7',
 ...}

Is it because the numbers are no strings? Can I see what in df.GeneID is stored? If it is a number or a string.. Could this cause the error? Or why isn't it replacing it properly?

jezrael · Accepted Answer · 2020-04-29T08:51:25.920

1

Possible problem is traling whitespaces, remove them by str.strip(), another problem is value from list not exist in column GeneID, so created NaNs:

df["GeneID"] = df["GeneID"].str.strip().map(final_dictionary1)

Or if possible some values not matched and need no replacement of this values:

df["GeneID"] = df["GeneID"].str.strip().replace(final_dictionary1)

edited Apr 29 '20 at 08:51

answered Apr 29 '20 at 08:46

jezrael

822,522
95
1,334
1,252

the second one gives me: TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType' – Anja Apr 29 '20 at 08:49
@Anja - What is `print (df["GeneID"].dtype)` ? – jezrael Apr 29 '20 at 08:50
object @jezrael – Anja Apr 29 '20 at 08:50
@Anja - Edited answer. – jezrael Apr 29 '20 at 08:53
This seems to be the right way.. but my notebook is running out of memory.. as it has 1097360 entries in df and 300000 entries in dictionary – Anja Apr 29 '20 at 09:06
@Anja - Merge is not possible use like [this](https://stackoverflow.com/q/53010406) ? – jezrael Apr 29 '20 at 09:07
Takes 5 minutes and then it deletes every variable I had before.. I think it's a memory problem . What do you thinn @jezrael? – Anja Apr 29 '20 at 09:08
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/212769/discussion-between-anja-and-jezrael). – Anja Apr 29 '20 at 09:16

Why does replacing dataframe columns with dictionary just create nan's?

replacing it with final_dictionary1

1 Answers1