How can I use a dictionary to do multiple search-and-replace operations?

Question

I have to replace text like "north", "south", etc. with "N", "S" etc. in address fields. I thought of making a dictionary to hold the replacements. Suppose we have:

replacements = {'NORTH':'N','SOUTH':'S','EAST':'E','WEST':'W'}
address = "123 north anywhere street"

Can I use the replacements dictionary to do all the replacements, for example by iterating over it? What would the code for this look like?

Answer 1

    address = "123 north anywhere street"
    
    for word, initial in {"NORTH": "N", "SOUTH": "S"}.items():
        address = address.replace(word.lower(), initial)
    print(address)

nice and concise and readable too.

Answer 2

you are close, actually:

dictionary = {"NORTH":"N", "SOUTH":"S" } 
for key in dictionary.iterkeys():
    address = address.upper().replace(key, dictionary[key])

Note: for Python 3 users, you should use .keys() instead of .iterkeys():

dictionary = {"NORTH":"N", "SOUTH":"S" } 
for key in dictionary.keys():
    address = address.upper().replace(key, dictionary[key])

Answer 3

One option I don't think anyone has yet suggested is to build a regular expression containing all of the keys and then simply do one replace on the string:

>>> import re
>>> l = {'NORTH':'N','SOUTH':'S','EAST':'E','WEST':'W'}
>>> pattern = '|'.join(sorted(re.escape(k) for k in l))
>>> address = "123 north anywhere street"
>>> re.sub(pattern, lambda m: l.get(m.group(0).upper()), address, flags=re.IGNORECASE)
'123 N anywhere street'
>>> 

This has the advantage that the regular expression can ignore the case of the input string without modifying it.

If you want to operate only on complete words then you can do that too with a simple modification of the pattern:

>>> pattern = r'\b({})\b'.format('|'.join(sorted(re.escape(k) for k in l)))
>>> address2 = "123 north anywhere southstreet"
>>> re.sub(pattern, lambda m: l.get(m.group(0).upper()), address2, flags=re.IGNORECASE)
'123 N anywhere southstreet'

Answer 4

You are probably looking for iteritems():

d = {'NORTH':'N','SOUTH':'S','EAST':'E','WEST':'W'}
address = "123 north anywhere street"

for k,v in d.iteritems():
    address = address.upper().replace(k, v)

address is now '123 N ANYWHERE STREET'

---

Well, if you want to preserve case, whitespace and nested words (e.g. Southstreet should not converted to Sstreet), consider using this simple list comprehension:

import re

l = {'NORTH':'N','SOUTH':'S','EAST':'E','WEST':'W'}

address = "North 123 East Anywhere Southstreet    West"

new_address = ''.join(l[p.upper()] if p.upper() in l else p for p in re.split(r'(\W+)', address))

new_address is now

N 123 E Anywhere Southstreet    W

Answer 5

"Translating" a string with a dictionary is a very common requirement. I propose a function that you might want to keep in your toolkit:

def translate(text, conversion_dict, before=None):
    """
    Translate words from a text using a conversion dictionary

    Arguments:
        text: the text to be translated
        conversion_dict: the conversion dictionary
        before: a function to transform the input
        (by default it will to a lowercase)
    """
    # if empty:
    if not text: return text
    # preliminary transformation:
    before = before or str.lower
    t = before(text)
    for key, value in conversion_dict.items():
        t = t.replace(key, value)
    return t

Then you can write:

>>> a = {'hello':'bonjour', 'world':'tout-le-monde'}
>>> translate('hello world', a)
'bonjour tout-le-monde'

Answer 6

I would suggest to use a regular expression instead of a simple replace. With a replace you have the risk that subparts of words are replaced which is maybe not what you want.

import json
import re

with open('filePath.txt') as f:
   data = f.read()

with open('filePath.json') as f:
   glossar = json.load(f)

for word, initial in glossar.items():
   data = re.sub(r'\b' + word + r'\b', initial, data)

print(data)

Answer 7

def replace_values_in_string(text, args_dict):
    for key in args_dict.keys():
        text = text.replace(key, str(args_dict[key]))
    return text

Answer 8

If you're looking for a concise way, you can go for reduce from functools:

from functools import reduce

str_to_replace = "The string for replacement."
replacement_dict = {"The ": "A new ", "for ": "after "}

str_replaced = reduce(lambda x, y: x.replace(*y), [str_to_replace, *list(replacement_dict.items())])
print(str_replaced)

Answer 9

Try,

import re
l = {'NORTH':'N','SOUTH':'S','EAST':'E','WEST':'W'}

address = "123 north anywhere street"

for k, v in l.iteritems():
    t = re.compile(re.escape(k), re.IGNORECASE)
    address = t.sub(v, address)
print(address)

Answer 10

Both using replace() and format() are not so precise:

data =  '{content} {address}'
for k,v in {"{content}":"some {address}", "{address}":"New York" }.items():
    data = data.replace(k,v)
# results: some New York New York

'{ {content} {address}'.format(**{'content':'str1', 'address':'str2'})
# results: ValueError: unexpected '{' in field name

It is better to translate with re.sub() if you need precise place:

import re
def translate(text, kw, ignore_case=False):
    search_keys = map(lambda x:re.escape(x), kw.keys())
    if ignore_case:
        kw = {k.lower():kw[k] for k in kw}
        regex = re.compile('|'.join(search_keys), re.IGNORECASE)
        res = regex.sub( lambda m:kw[m.group().lower()], text)
    else:
        regex = re.compile('|'.join(search_keys))
        res = regex.sub( lambda m:kw[m.group()], text)

    return res

#'score: 99.5% name:%(name)s' %{'name':'foo'}
res = translate( 'score: 99.5% name:{name}', {'{name}':'foo'})
print(res)

res = translate( 'score: 99.5% name:{NAME}', {'{name}':'foo'}, ignore_case=True)
print(res)

Answer 11

All of these answers are good, but you are missing python string substitution - it's simple and quick, but requires your string to be formatted correctly.

address = "123 %(direction)s anywhere street"
print(address % {"direction": "N"})

Answer 12

A faster way to handle this would be to respect word boundaries and look up each token in your dictionary only once:

token_mapping = {
    'north': 'N', 'south': 'S', 
    'east': 'E', 'west': 'W'
    'street': 'St',
    }

def tokenize(text):
    return text.lower().split()

def detokenize(tokens):
    return ' '.join(tokens)

def replace_tokens(text, token_mapping=token_mapping):
    input_tokens = tokenize(text)
    output_tokens = []
    for tok in input_tokens:
        output_tokens.append(token_mapping.get(tok, tok))
    return detokenize(output_tokens)

>>> replace_tokens("123 north anywhere street")
'123 N anywhere St'

Another advantage of this approach is that you can fold the case of individual tokens to suit your needs:

def detokenize(tokens):
    return ' '.join([t.title() for t in tokens])

>>> replace_tokens("123 north anywhere street")
'123 N Anywhere St'

This is the approach used by web-scale NLP, including spelling-correctors and abbreviation expanders/contractors.

Answer 13

The advantage of Duncan's approach is that it is careful not to overwrite previous answers. For example if you have {"Shirt": "Tank Top", "Top": "Sweater"}, the other approaches replace "Shirt" with "Tank Sweater".

The following code extends that approach, but sorts the keys such that the longest one is always found first and it uses named groups to search case insensitively.

import re
root_synonyms = {'NORTH':'N','SOUTH':'S','EAST':'E','WEST':'W'}
# put the longest search term first. This menas the system does not replace "top" before "tank top"
synonym_keys = sorted(root_synonyms.keys(),key=len,reverse=True)
# the groups will be named w1, w2, ... . Determine what each of them should become
number_mapping = {f'w{i}':root_synonyms[key] for i,key in enumerate(synonym_keys) }
# make a regex for each word where "tank top" or "tank  top" are the same
search_terms = [re.sub(r'\s+',r'\s+',re.escape(k)) for k in synonym_keys]
# give each search term a name w1 etc where
search_terms = [f'(?P<w{i}>\\b{key}\\b)' for i,key in enumerate(search_terms)]
# make one huge regex
search_terms = '|'.join(search_terms)
# compile it for speed
search_re = re.compile(search_terms,re.IGNORECASE)

query = "123 north anywhere street"
result = re.sub(search_re,lambda x: number_mapping[x.lastgroup],query)
print(result)