Python-Counting element frequency in a 2D list

Question

I want to know if there is a way to count element frequencies in a 2D python list. For 1D lists, we can use

list.count(word)

but what if I have a list:

a = [ ['hello', 'friends', 'its', 'mrpycharm'], 
      ['mrpycharm', 'it', 'is'], 
      ['its', 'mrpycharm'] ]

can i find the frequency for each word in this 2D list?

score 7 · Accepted Answer · answered Oct 19 '16 at 19:36

7

Assuming I understand what you want,

>>> collections.Counter([x for sublist in a for x in sublist])
Counter({'mrpycharm': 3, 'its': 2, 'friends': 1, 'is': 1, 'it': 1, 'hello': 1})

Or,

>>> c = collections.Counter()
>>> for sublist in a:
...     c.update(sublist)
...
>>> c
Counter({'mrpycharm': 3, 'its': 2, 'friends': 1, 'is': 1, 'it': 1, 'hello': 1})

answered Oct 19 '16 at 19:36

sberry

128,281
18
138
165

Do you know if using a generator expression instead of a list comprehension would result in a speedup? `Counter(x for sublist in a for x in sublist)` – Patrick Haugh Oct 19 '16 at 20:10

Patrick Haugh · Answer 2 · 2016-10-19T19:38:18.077

4

You can use a defaultdict:

from collections import defaultdict
d = defaultdict(int)
for sublist in a:
    for word in sublist:
        d[word] += 1

edited Oct 19 '16 at 19:38

answered Oct 19 '16 at 19:37

Patrick Haugh

59,226
13
88
96

1

@juanpa.arrivillaga why is it a wrong collection? – Ahmed Dhanani Oct 19 '16 at 19:46
Look at sberry's answer – juanpa.arrivillaga Oct 19 '16 at 19:47
but this seems to work fine too – Ahmed Dhanani Oct 19 '16 at 19:48
1

@juanpa.arrivillaga: But that doesn't makes it the wrong collection. – Moinuddin Quadri Oct 19 '16 at 19:49
It works fine. A normal `dict` would also work fine too. But if you are already going to import from collections, why not use the collection *made for counting elements*? – juanpa.arrivillaga Oct 19 '16 at 19:51
2

Not the wrong collection, just a different choice. Prior to its introduction in 2.7 this is exactly how I would have done it. For what it's worth, for this size data on my machine (and CPython2.7) this solution is more than 50% faster than using Counter. – sberry Oct 19 '16 at 20:02

Moinuddin Quadri · Answer 3 · 2016-10-19T19:51:56.240

You already know about list.count(). Simply get the count of word in each sublist and sum them. For example:

>>> my_word = 'its'
>>> sum(sublist.count(my_word) for sublist in a)
2

In case you want the frequency of every word present in your list, there are many good answers available here for that. Alternatively, in case you want to do it without any import (using normal dict), you may do:

my_dict = {}
for sublist in a:
    for item in sublist:
        if item not in my_dict:
            my_dict[item] = 0
        my_dict[item] += 1

# Value of my_dict:
{'friends': 1, 'is': 1, 'it': 1, 'its': 2, 'mrpycharm': 3, 'hello': 1}

Python-Counting element frequency in a 2D list

3 Answers3

Linked

Related