2

The function below should return a new dictionary which has summed the values up. 

import functools
def sumUp(d):
    for k in d:
        d.update({k: functools.reduce(lambda x, y: x + y, d[k])})
    print(d)

When i call the function as follows i get the following TypeError which i can't understand why:

sumUp({"Ungefucht": (165, 165, 165, 255, 286.25, 255, 165, 240, 240, 150), "Malsch": (120, 240, 120, 120, 120, 120, 120), "AAA": (1, 2), "Fens": (115.20, 69.60, 28.80, 50.40), "Betti": (82.50,)})

Traceback (most recent call last):

File "", line 1, in

File "/home/amir/programming/python/lern.py", line 6, in sumUp print(d)

TypeError: reduce() arg 2 must support iteration

When i omit one of the key-values it works fine:

sumUp({"Ungefucht": (165, 165, 165, 255, 286.25, 255, 165, 240, 240, 150), "Malsch": (120, 240, 120, 120, 120, 120, 120), "AAA": (1, 2), "Fens": (115.20, 69.60, 28.80, 50.40)})

{'Malsch': 960, 'Ungefucht': 2086.25, 'Fens': 264.0, 'AAA': 3}

Why is the first example with one more item not working as expected?

Dimitris Fasarakis Hilliard
  • 150,925
  • 31
  • 268
  • 253
  • 1
    This is very strange. If you add an item after Betti it works – EoinS Sep 27 '16 at 20:32
  • 1
    It seems it has problems if there are 5 key-val pairs! –  Sep 27 '16 at 20:35
  • 1
    @Bharel seems to have found a way to do what you want to do but I am interested in finding out exactly why this is happening. Checking key hashes now – EoinS Sep 27 '16 at 20:41
  • `for k,v in d.items():d.update({k: functools.reduce(lambda x, y: x + y, v)})` or `for k in list(d)` would work I imagine, changing the dict as you iterate over it is causing the error – Padraic Cunningham Sep 27 '16 at 20:53
  • @PadraicCunningham It looks like the iterator gets messed up during update. Can it be because of an order change? That's why I thought key hashes caused the original problem or dictionary insertion order. But then again, it's the same key... – Bharel Sep 27 '16 at 21:00
  • @PadraicCunningham Also, why would it throw an error in the reduce function? The error in this case should be a RuntimeError on the for loop. – Bharel Sep 27 '16 at 21:02
  • @Bharel, I bet if you make all the values inside the tuples strings it would work, what is happening is reduce is trying to reduce an integer because pointers are getting skewed mutating the data as you iterate over it. Basically somewhere something like `functools.reduce(sum, 4))` is happening. – Padraic Cunningham Sep 27 '16 at 21:04
  • @PadraicCunningham Still though, why would the pointers skip exactly there out of all places? – Bharel Sep 27 '16 at 21:12
  • @Bharel, it happens on different iterations using python3, sometimes it does not happen at all. – Padraic Cunningham Sep 27 '16 at 21:16
  • @PadraicCunningham So I was correct in my first assumption. The dict is updated, causing the iteration order to change due to the salting in Py3, and on the next iteration, it gets to a key already computed by the reduce function and thus gets an int. – Bharel Sep 27 '16 at 21:20
  • 1
    yes, all bets are off when you iterate over and mutate any container. If you do use strings you will most likely see some tuples are missed completely even though there is no error. – Padraic Cunningham Sep 27 '16 at 21:22
  • @Padraic also I guess it doesn't cause a runtime error due to the fact the count doesn't change, only the order – Bharel Sep 27 '16 at 21:26
  • @Jim it's not a duplicate mate. Dictionary update and item insertion are different. – Bharel Sep 27 '16 at 21:33
  • @PadraicCunningham Hang on. I just figured out that I'm running it using Python 3.6. In Python 3.6 I was not able to reproduce it. Why is that? – Bharel Sep 27 '16 at 21:33
  • The dup linked is one of the many questions dealing with the essential part of this question, namely, altering a mutable structure while you iterate over it. This behavior is documented as something you *should not do*. – Dimitris Fasarakis Hilliard Sep 27 '16 at 21:34
  • @JimFasarakis-Hilliard Does not happen in Python 3.6 though. It's not a usual behavior of mutation. – Bharel Sep 27 '16 at 21:35
  • It appears that using `{}.update` causes the key order to change and screwing up the data structures inside the dict. Try inserting `print(k,d[k], type(d[k]))` inside the loop above the `reduce` – dawg Sep 27 '16 at 21:36
  • @Bharel I'm guessing that's another side effect of the new implementation of dictionaries in `3.6` (and, most likely, due to the insertion order being maintained). The new dictionary implementation is, of course, not the subject of this question. – Dimitris Fasarakis Hilliard Sep 27 '16 at 21:42
  • @JimFasarakis-Hilliard Aye, just checked it. – Bharel Sep 27 '16 at 21:46
  • 1
    @Bharel, Jim is correct, there are a lot of new changes in 3.6 http://blog.python.org/2016/09/python-core-development-sprint-2016-36.html – Padraic Cunningham Sep 27 '16 at 21:53

1 Answers1

2

This works:

def sumUp(d):
    new_d = {k: sum(v) for k, v in d.items()}
    print(new_d)
    return new_d

Keep in mind you are updating the dictionary while iterating over it, which causes all sorts of strange behavior.

The behavior is quite random and depends on the salting of the key's hashes (which modifies the dict order). You can't reproduce it consistently in Python 3.

This is your code fixed:

def sumUp2(d):
    for k in d:
        d[k] = functools.reduce(lambda x, y: x + y, d[k])
    print(d)

It sets the key instead of updating the dict, which is safe.

Bharel
  • 23,672
  • 5
  • 40
  • 80