4

I want to update the keys of my dictionary c with its new keys k_new. Even though I am referring different stack overflow questions like this it does not get updated. Please tell me where I make it wrong.

from nltk.stem import WordNetLemmatizer
lemmatizer = WordNetLemmatizer()
c = {'apples': 3, 'biscuits and tea': 3, 'oranges and onions': 4}
for k in c:
    splits=k.split()
    k_new= " ".join(lemmatizer.lemmatize(w.lower()) for w in splits)
    c[k_new] = c.pop(k)
print(c)

PS: I also used:

c[k_new] = c[k]
del c[k]

Then I get RuntimeError: dictionary changed size during iteration

Please help me

Willem Van Onsem
  • 443,496
  • 30
  • 428
  • 555

2 Answers2

3

You update the dictionary while you iterate over it:

from nltk.stem import WordNetLemmatizer
lemmatizer = WordNetLemmatizer()
c = {'apples': 3, 'biscuits and tea': 3, 'oranges and onions': 4}
for k in c:  # iterate over c
    splits=k.split()
    k_new= " ".join(lemmatizer.lemmatize(w.lower()) for w in splits)
    c[k_new] = c.pop(k)  # update (two times) c
print(c)

updating a collection while you iterate over it is usually a very bad idea. Most data structures are not designed to handle this.

You can however construct a new dictionary:

from nltk.stem import WordNetLemmatizer
lemmatizer = WordNetLemmatizer()
c = {'apples': 3, 'biscuits and tea': 3, 'oranges and onions': 4}
c_new = {}
for k in c:
    splits=k.split()
    k_new= " ".join(lemmatizer.lemmatize(w.lower()) for w in splits)
    c_new[k_new] = c[k]
print(c_new)

We can make this more elegant by using dictionary comprehension:

{" ".join(lemmatizer.lemmatize(w.lower()) for w in k.split()): v
 for k,v in c.items()}

this one-liner constructs a new dictionary where we iterate over key-value pairs k,v of c, and add a key " ".join(lemmatizer.lemmatize(w.lower()) for w in k.split()) that we associate with the value v.

Willem Van Onsem
  • 443,496
  • 30
  • 428
  • 555
  • @thebjorn: yes. That is what I do in the list comprehension: I propose a fix, but then aim to show an - imho - more elegant way to do it. – Willem Van Onsem Oct 04 '17 at 08:32
  • both your solutions create new dicts, which is fine but not strictly necessary in this case since the OP can mutate his dict in place if he iterates over `c.items()` (or creates any other separate data structure to hold the keys). Personally I think the dict comprehension is too complex in this case (and the `: v` almost disappears), but that's probably more of a taste issue.. – thebjorn Oct 04 '17 at 08:36
  • @thebjorn: but then we will use memory to hold the keys. `c.items()` is an iterable as well, so this is not a solution either (gives the same error, in Python-3.x). Since `items()` is a proxy that iterates almost the same way over the dict, but takes the value in the dict as well into account. – Willem Van Onsem Oct 04 '17 at 08:39
  • that's a Py3 misfeature, in my world `c.items()` returns a list :-D ps: you're holding two entire copies of the dictionary in memory, so the `c.items()` (py2) or `list(c)`don't consume any more memory.. – thebjorn Oct 04 '17 at 08:43
  • @thebjorn: then how is the list maintained? It means a malloc to construct a list object, that has at least as size the size of a reference pointer times the number of items in it. Since one want the ammortized cost of an append to be *O(1)*, usually it can be as large as the next power of two. – Willem Van Onsem Oct 04 '17 at 08:43
2

Iterating over dictionary while it is changing might have weired effects since you are using a real-time reference to the keys. Just make a list out of the key view and it will work:

for k in list(c):
    ...
Klaus D.
  • 13,874
  • 5
  • 41
  • 48
  • Thank you for the answer. However, I want to get the values as well assigned for the keys while changing keys. Can we do it with lists? –  Oct 04 '17 at 08:57