Use a dictionary to replace words in a sentence

Question

I have created a dictionary that I use to use to bring a variety of words to its base form.

dictionary = {'sunny': 'sun', 'banking': 'bank'}

def stemmingWords(sentence, dictionary):
    for word in sentence.split():
        temp = []
        if word in dictionary:
            word = dictionary[word]
            temp.append(word)
    sentence = ' '.join(temp)
    return(sentence)

Now if print the separate words it seems to work. However when I insert a whole sentence and I would like an updated version of this sentence something seems to go wrong. For example if I do:

sentence = "the sun us shining"
new_sentence = stemmingWords(sentence, dictionary)
print(new_sentence)

Gives me "shining". While I am looking "the sunny in shining".

Any thoughts on what goes wrong here?

your dictionary would be more useful and efficient if it was as `{ 'sun':'sunny', 'bank' : 'banking'}` — RomanPerekhrest, Jan 29 '17 at 08:22
For the sentence : "the sun is shining"....What is ur expected output? — Vijay Kalmath, Jan 29 '17 at 08:26
@VijayKalmath, the output I expect is "the sunny is shining". So sun should be replaced by sunny... (thats what the dictionary is for) — Frits Verstraten, Jan 29 '17 at 08:40

Jean-François Fabre · Answer 1 · 2017-01-29T09:55:05.907

5

First, your dictionary is the wrong way round, reverse it

dictionary = {'sunny': 'sun', 'banking': 'bank'}

a simple way to do it to avoid retyping it would be:

dictionary = {v:k for k,v in dictionary.items()}

note that if several words match a same word, reverting the dictionary won't work you have to solve the ambiguity first: so manually:

dictionary = {'sun', 'sunny': , 'sunn' : 'sunny', 'bank': 'banking'}

Then split and rebuild the string using a list comprehension and a get access returning the original value if not in the dictionary

def stemmingWords(sentence,dictionary):
    return " ".join([dictionary.get(w,w) for w in sentence.split()])

print(stemmingWords("the sun is shining",dictionary))

result:

the sunny is shining

note the deliberate ([]) when using join. It's faster to pass explicitly the list comprehension than the generator in that case.

edited Jan 29 '17 at 09:55

answered Jan 29 '17 at 09:01

Jean-François Fabre

137,073
23
153
219

Very pythonic answer – jlandercy Jan 29 '17 at 09:12
Many thanks! However the dictionary should be able to have multiple values per key. So like dictionary['sunny'] = ['sun', 'sunn']. How would you do this?> – Frits Verstraten Jan 29 '17 at 09:31
You don't get the principle of a dictionary. key comes first, then the matching value. You write it backwards. Check my edit. – Jean-François Fabre Jan 29 '17 at 09:53

score 0 · Answer 2 · edited May 23 '17 at 12:01

The issues with your functions are the followig:

Dictionary key-value are swapped.
Workaround: Try https://stackoverflow.com/a/1031878/4954434
The append() function by default will append to end of the list.
Thus position is not taken care properly.
temp does not have all the words.

The following function should work.

def stemmingWords(sentence, dictionary):
    dictionary = dict((v,k) for k,v in dictionary.iteritems())

    splitted = sentence.split()
    for i in range(len(splitted)):
        if splitted[i] in dictionary:
            print splitted[i]
            splitted[i] = dictionary[splitted[i]]
    sentence = ' '.join(splitted)
    return(sentence)

While I hope this answer will help newbies, Jean-François Fabre's answer is far better.

Use a dictionary to replace words in a sentence

2 Answers2