How to replace text corpus using a matching word from dictionary using Regex

Question

I am trying standardize my address data which present inside dataframe.

I was only able correctly replace words where in dictionary where one word has only one values. I am not noble to replace when one word has mutliple values .

Input data

data['Postal_Addr']

UNIT 3510 35/F THE CENTRE 
HARNEYS THE CENTRE 99 QUEENS STR ROAD CETRAL
OCBC WING HANG BANK UNIT B 5/FLR WING HANG INSURANCE CENTRE BUILDING
M3 CAPITAL PARTNERS 50 LEVEL 3 CENTRAL
GALERIE H LDT 50  CENTRAL 17TH FLOOR 
36/FL INFINITUS PLAZA STREET 
51 SAI ST STRAT RAOD

Dictionary

Mapping_matrix.set_index('Orginal')['Related_Words'].to_dict()

{'FLOOR ': 'F,FL,FLR',
 'ROAD': 'RD, RD.,RAOD',
 'STREET': 'ST, STR,STRAT'
 'CENTRAL': 'CENTRE, CETRAL'}

Expected Output

UNIT 3510 35/FLOOR THE CENTRAL 99 QUEENS ROAD CENTRAL 
HARNEYS THE CENTRAL 99  ROAD CENTRAL
OCBC WING HANG BANK UNIT B 5/FLOOR WING HANG INSURANCE CENTRAL BUILDING
M3 CAPITAL PARTNERS 50 LEVEL 3 CENTRAL
GALERIE  LDT 50 CENTRAL 17TH FLOOR 
51 SAI STREET STREET ROAD

I tried code mentionedrefrence(How to replace a string using a dictionary containing multiple values for a key in python)

def replace_words(d, col):
    d1={r'(?<!\S)'+ k.strip() + r'(?!\S)':k1 for k1, v1 in d.items() for k in v1.split(',')}
    df[col] = df[col].replace(d1, regex=True)
    return df[col]

df['col'] = replace_words(d, 'col')

But it didnt work.

@jezrael, still didnt work, its giving an error `'Series' object has no attribute 'split'` — Snehal pankaj, Jan 05 '18 at 11:35
Do you try `d = Mapping_matrix.set_index('Orginal')['Related_Words'].to_dict()`? — jezrael, Jan 05 '18 at 11:36
No, you need create dictionaryu first and then apply solution. so `d = Mapping_matrix.set_index('Orginal')['Related_Words'].to_dict‌()` and then `data['Postal_Addr'] = replace_words(d, 'Postal_Addr')` — jezrael, Jan 05 '18 at 11:39
@jezrael, I had made `df` to `data`. I didnt assign d properly.Now its running.I have around 60000 rows to update. will this take lot of time. — Snehal pankaj, Jan 05 '18 at 11:45
Yes, it is not trivial operation, so it is expected, unfortunately :( — jezrael, Jan 05 '18 at 11:46
@jezrael, failed due to an `regex error`- `missing ), unterminated subpattern at position 7` — Snehal pankaj, Jan 05 '18 at 11:49
It s not good. It means 1. some regex problem, because not escaped character in key of dictinary. 2. some value in column `Postal_Addr` with bad data. And not easy find it, because data related issue :( — jezrael, Jan 05 '18 at 11:52
@jezrael, this happen only when we have special characters right. Do you have better solution — Snehal pankaj, Jan 05 '18 at 11:54
Hmmm, there is one idea. change `r'(?<!\S)'+ k.strip() + r'(?!\S)'` to `r'(?<!\S)'+ re.escape(k.strip()) + r'(?!\S)'` and before add `import re` — jezrael, Jan 05 '18 at 11:59
@jezrael, Its running now.. I think you update the answer and remove duplicate — Snehal pankaj, Jan 05 '18 at 12:13
I really like do it, but people in SO kill me for it, so I cant do it. Sorry :( — jezrael, Jan 05 '18 at 12:14
@jezrael,It ran successfully, Update the original the answer. — Snehal pankaj, Jan 05 '18 at 12:22

How to replace text corpus using a matching word from dictionary using Regex

0 Answers0