4

I am aware of Merge dictionaries without overwriting values, merging "several" python dictionaries, How to merge multiple dicts with same key?, and How to merge two Python dictionaries in a single expression?.

My problem is slightly different, however.

Lets say I have these three dictionaries

dict_a = {'a': [3.212], 'b': [0.0]}
dict_b = {'a': [923.22, 3.212], 'c': [123.32]}
dict_c = {'b': [0.0]}

The merged result should be

result_dict = {'a': [3.212, 3.212, 923.22], 'b': [0.0, 0.0], 'c': [123.32]}

This code here works, but would nest lists within the lists

result_dict = {}
dicts = [dict_a, dict_b, dict_c]

for d in dicts:
    for k, v in d.iteritems():
        result_dict.setdefault(k, []).append(v)

Using extend instead of append would prevent the nested lists, but doesn't work if a key doesn't exist yet. So basically it should do a update without the overwriting, as in the other threads.

Sorry, it was a mistake on my side. It was too late yesterday and I didn't notice the line that threw the error wasn't the one I thought it did, therefore assumed my dictionaries would already have the above structure. In fact, mgilson was correct assuming that it was related to a TypeError. To be exact, an 'uniterable float'.

Community
  • 1
  • 1
cherrun
  • 2,102
  • 8
  • 34
  • 51
  • 2
    Could you explain why `extend` won't work? Shouldn't `setdefault(k, [])` take care of the nonexistent key problem? – DSM Dec 11 '13 at 02:45
  • 1
    why did the result list for `a` end up in that order? I would have expected: `[ 3.212, 923.22, 3.212]` – mgilson Dec 11 '13 at 02:45
  • @mgilson Yes, sorry. I already sorted in my head. But it should be the way you said. – cherrun Dec 11 '13 at 09:42

1 Answers1

4

I'm pretty sure that .extend works here ...

>>> dict_a = {'a': [3.212], 'b': [0.0]}
>>> dict_b = {'a': [923.22, 3.212], 'c': [123.32]}
>>> dict_c = {'b': [0.0]}
>>> result_dict = {}
>>> dicts = [dict_a, dict_b, dict_c]
>>> 
>>> for d in dicts:
...     for k, v in d.iteritems():
...         result_dict.setdefault(k, []).extend(v)
... 
>>> result_dict
{'a': [3.212, 923.22, 3.212], 'c': [123.32], 'b': [0.0, 0.0]}

The magic is in the dict.setdefault method. If the key doesn't exist, setdefault adds a new key with the default value you provide. It then returns that default value which can then be modified.

Note that this solution will have a problem if the item v is not iterable. Perhaps that's what you meant? e.g. if dict_a = {'a': [3.212], 'b': 0.0}.

In this case, I think you'll need to resort to catching the TypeError: type object is not iterable in a try-except clause:

for d in dicts:
    for k, v in d.iteritems():
        try:
            result_dict.setdefault(k, []).extend(v)
        except TypeError:
            result_dict[k].append(v)
mgilson
  • 300,191
  • 65
  • 633
  • 696
  • I haven't been following all the stuff in `abc`. Is there anything which is basically "scalar", i.e. "not iterable, or iterable but a string"? – DSM Dec 11 '13 at 02:55
  • @DSM -- Not sure I get you question. Not sure I get OP's question. Everything in OP's sample input is iterable, but OP said that `.extend` wouldn't work for some reason which made me think that OP had tried it (though maybe not). So, I started grasping for a reason *why* `.extend` may have failed. A non-iterable item seemed to be the most likely culprit... (but just a guess). – mgilson Dec 11 '13 at 02:58
  • I only mean that the TypeError approach won't prevent scalarlike but iterable objects, such as strings, from taking the `.extend` branch and turning `"Fred"` into `"F", "r", "e", "d"`. – DSM Dec 11 '13 at 03:04
  • Ahh ... You're referencing the fact that `str` has no `.__iter__` function? Fortunately python3.x fixed that problem ... and I think that strings are the only object commonly treated as scalars that are also iterable... – mgilson Dec 11 '13 at 03:07
  • Man, I'm not doing a very good job of explaining myself. :^) No python 3 connection, only that this code will work for scalar number values but won't work for scalar strings, and I was wondering if there was some clever trick other than `not isinstance(v, basestring)` that I was the last one to hear about, like usual. – DSM Dec 11 '13 at 03:11
  • @DSM -- Nope, no trick (other than `hasattr(v, '__iter__')` on python2.x). If you want to treat strings like scalars, then you need to special case it. – mgilson Dec 11 '13 at 03:15
  • @mgilson Good assumption. I edited my first post. It was indeed a `TypeError`. Your solution works fine, but what I don't get is, if I try that in the interpreter, it gives me `KeyError`, when I try to do `result_dict['foo'].append(3.14)`. Could you explain why this is? It seems to work in my Python script. – cherrun Dec 11 '13 at 09:51