Python Collections Counter for a List of Dictionaries

Question

I have a dynamically growing list of arrays that I would like to add like values together. Here's an example:

{"something" : [{"one":"200"}, {"three":"400"}, {"one":"100"}, {"two":"800"} ... ]}

I'd like to be able to add together the dictionaries inside the list. So, in this case for the key "something", the result would be:

["one":400, "three": 400, "two": 800]

or something to that effect. I'm familiar with the Python's collection counter, but since the "something" list contains dicts, it will not work (unless I'm missing something). The dict is also being dynamically created, so I can't build the list without the dicts. EG:

Counter({'b':3, 'c':4, 'd':5, 'b':2})

Would normally work, but as soon as I try to add an element, the previous value will be overwritten. I've noticed other questions such as these:

Is there any pythonic way to combine two dicts (adding values for keys that appear in both)?

Python count of items in a dictionary of lists

But again, the objects within the list are dicts.

your structure is somewhat confusing; You have a single outer dictionary whose values are lists of dictionaries with only one key each? Are there many actual entries in the outer dictionary we have to worry about? Or are we just adding the dictionaries in the inner list for the `"something"` key? A more complete example of the expected input and output would be helpful. — aruisdante, Feb 09 '15 at 02:34
@aruisdante sorry for the confusion. The structure is very frustrating but I don't have control over it other than modifying it. To answer your questions: 1) Yes, only one key each. 2) There may be other dictionaries such as Steve Jessop's answer illustrates. — Donato Perconti, Feb 09 '15 at 04:25

Steve Jessop · Accepted Answer · 2015-02-09T23:51:18.160

I think this does what you want, but I'm not sure because I don't know what "The dict is also being dynamically created, so I can't build the list without the dicts" means. Still:

input = {
    "something" : [{"one":"200"}, {"three":"400"}, {"one":"100"}, {"two":"800"}], 
    "foo" : [{"a" : 100, "b" : 200}, {"a" : 300, "b": 400}],
}

def counterize(x):
    return Counter({k : int(v) for k, v in x.iteritems()})

counts = {
    k : sum((counterize(x) for x in v), Counter()) 
    for k, v in input.iteritems()
}

Result:

{
    'foo': Counter({'b': 600, 'a': 400}), 
    'something': Counter({'two': 800, 'three': 400, 'one': 300})
}

I expect using sum with Counter is inefficient (in the same way that using sum with strings is so inefficient that Guido banned it), but I might be wrong. Anyway, if you have performance problems, you could write a function that creates a Counter and repeatedly calls += or update on it:

def makeints(x):
    return {k : int(v) for k, v in x.iteritems()}

def total(seq):
    result = Counter()
    for s in seq:
        result.update(s)
    return result

counts = {k : total(makeints(x) for x in v) for k, v in input.iteritems()}

Marcin · Answer 2 · 2015-02-09T02:49:59.960

One way would be do as follows:

from collections import defaultdict

d = {"something" :
     [{"one":"200"}, {"three":"400"}, {"one":"100"}, {"two":"800"}]}

dd = defaultdict(list)

# first get and group values from the original data structure
# and change strings to ints
for inner_dict in d['something']:
    for k,v in inner_dict.items():
        dd[k].append(int(v))


# second. create output dictionary by summing grouped elemetns
# from the first step.
out_dict =  {k:sum(v) for k,v in dd.items()}

print(out_dict)
# {'two': 800, 'one': 300, 'three': 400}

In here I don't use counter, but defaultdict. Its a two step approach.

Python Collections Counter for a List of Dictionaries

2 Answers2

Linked