I'm building dictionary in python from a table which is ~2Gb big. It includes ID and description. eg:
id | description |
---|---|
COG5157 | cell division cycle 73 |
COG5153 | Lipase |
COG5156 | complex subunit 10 |
COG5151 | Transcription factor |
and I also have another file with IDs, which I would like to map on this table (to obtain description) I have approximately 140000 IDs
I'm running it in python3 for
loop, which is actually pretty slow (up to 90 minutes).
nogidsdict=nog_db.set_index('id')['description'].to_dict()
prof_db['description']=""
for i in range(len(prof_id)):
try:
prof_db.iloc[i,2]=nogidsdict[prof_id[i]]
except:
prof_db.iloc[i,2]='None'
Is it possible to obtain values from dictionary for batch of keys without looping? Do you have any suggestion how I could speed up this part of script?
Thank you in advance!