Counting occurences of words that appear in a list using Python

Question

I have appended excel sheet values to a list using xlrd. I called the list a_master. I have a text file with words that I want to count the occurrences of that appear in this list (I called this file dictionary and theirs 1 word per line). Here is the code:

with open("dictionary.txt","r") as f:
for line in f:
    print "Count " + line + str((a_master).count(line))

For some reason though, the count comes back with zero for every count word that exists in the text file. If I write out the count for one of these words myself:

 print str((a_master).count("server"))

It counts the occurrences no problem.I have also tried

print line

in order to see if it is seeing the words in the dictionary.txt file correctly and it is.

This is a duplicate of http://stackoverflow.com/questions/8742732/python-number-of-word-occurrences — tommy.carstensen, Apr 23 '15 at 08:36
@tommy.carstensen: Not an exact duplicate. Please read my answer. — Abhijit, Apr 23 '15 at 08:37

Abhijit · Accepted Answer · 2015-04-23T08:33:36.310

1

Lines read from the file is terminated by newline character. There may also be white space at the end. It is better to strip out any whitespace before doing a lookup

with open("dictionary.txt","r") as f:
    for line in f:
        print "Count " + line + str((a_master).count(line.strip()))

Note Ideally, searching a list is linear and may not be optimal in most cases. I think collections.Counter is suitable for situation as you depicted.

Re-interpret your list as a dictionary where the key is the item and the value is the occurrence by passing it through collections.Counter as shown below

a_master = collections.Counter(a_master)

and you can re-write your code as

from itertools import imap
with open("dictionary.txt","r") as f:
    for line in imap(str.strip, f):
        print "Count {} {}".format(line, a_master[line])

edited Apr 23 '15 at 08:33

answered Apr 23 '15 at 08:28

Abhijit

62,056
18
131
204

@Danny: If this answers your question, please consider accepting it. – Abhijit Apr 23 '15 at 08:34
One more question actually. In order to count occurrences of words in the list, I split up each word in the list. But what if i wanted to count the occurrences of "software defined networking". How would i go about doing this? Would i need to rewrite how that sentence appears in the dictionary.txt file? My txt file has a mixture of both single words as well as sentences. – Danny Apr 23 '15 at 08:42
@Danny: Your second question is too complex to be answered in the comment box. I would suggest you to post a new question and someone may want to answer it. Just to brief you here, search for trie and prefix tree. That might be the solution to your problem. – Abhijit Apr 23 '15 at 08:49

score 0 · Answer 2 · answered Apr 23 '15 at 08:34

0

Use collections.Counter():

import re
import collections
words = re.findall(r'\w+', open('dictionary.txt').read().lower())
collections.Counter(words)

Why is this question tagged xlrd by the way?

answered Apr 23 '15 at 08:34

tommy.carstensen

8,962
15
65
108

Counting occurences of words that appear in a list using Python

2 Answers2