1

I've created a simple word count program in python which reads a text file, counts the word frequency and writes the result to another file. The problem is when the word gets repeated, the program writes the initial as well as final count of the same word. For Example, if a word "hello" is repeated say 3 times, the program writes 3 instance of hello in output as :

Word - Frequency Count

hello - 1

hello - 2

hello - 3

The code is:

counts ={}
for w in words:
 counts[w] = counts.get(w,0) + 1
 outfile.write(w+','+str(counts[w])+'\n')'

Any help would be appreciated. I'm very much new in python.

sulav_lfc
  • 772
  • 2
  • 14
  • 34
  • 2
    First count, then print. Don't print while counting. So put the last line in its own loop, where you loop over the counts. – towr Feb 19 '14 at 07:14
  • well in that case, i'm not being able to extract words – sulav_lfc Feb 19 '14 at 07:16
  • related: [Sorted Word frequency count using python](http://stackoverflow.com/a/4088296/4279) – jfs Feb 19 '14 at 07:20

3 Answers3

5

The actual way to solve this is to use Counter, like this:

>>> from collections import Counter
>>> words = ['b','b','the','the','the','c']
>>> Counter(words).most_common()
[('the', 3), ('b', 2), ('c', 1)]

The other way to solve it, is by using a defaultdict, which will work just like the Counter example above:

>>> from collections import defaultdict
>>> d = defaultdict(int)
>>> for word in words:
...    d[word] += 1
...
>>> d
defaultdict(<type 'int'>, {'the': 3, 'b': 2, 'c': 1})

No matter how you count the words, you can only write to the file once all words are counted; otherwise you are writing once for each "count", and as soon as the word appears more than once, you will have doubled out your output.

So, first collect the counts, then write them out.

Burhan Khalid
  • 169,990
  • 18
  • 245
  • 284
1

The way to make the code work:

counts ={}
for w in words:
    counts[w] = counts.get(w,0) + 1

for w in counts:
    outfile.write(w+','+str(counts[w])+'\n')

But I think Burhan Khalid's suggestion of using Counter is a better way to solve the problem.

towr
  • 4,147
  • 3
  • 20
  • 30
0

Have you considered first storing the frequency count in your program, then writing it all at the end? It would certainly be simpler than rewriting the output file for every count.

itsmichaelwang
  • 2,282
  • 4
  • 16
  • 25