Words, sorted by frequency, in a book (.txt file)

Question

I'm using:

from collections import Counter
wordlist = open('mybook.txt','r').read().split()
c = Counter(wordlist)
print c

# result :
# Counter({'the': 9530, 'to': 5004, 'a': 4203, 'and': 4202, 'was': 4197, 'of': 3912, 'I': 2852, 'that': 2574, ... })

to print all the words of a book, sorted by frequency.

How to write this result in a .txt output file ?

g = open('wordfreq.txt','w')
g.write(c)   # here it fails

Here is the desired output wordfreq.txt :

the, 9530
to, 5004
a, 5004
and, 4203
was, 4197
...

what did you try to do? have you done any research? I hope nobody answers this and lets you actually try something. SO is not a palce to just get someone to code for you for free. You need to attempt it and research how to do something first. — John Ruddell, Nov 02 '15 at 21:51
@JohnRuddell if you don't feel answer it, then *don't do it.* I tried various things, including trying to `json.dumps` the `dict` then realizing it's not only a `dict` but a more complex thing... Well again, if you don't like this question, then don't answer it, but I don't see the point in your comment. — Basj, Nov 02 '15 at 22:02
@Basj few things, first I haven't downvoted your question.. but others have because your question shows no attempt to solve this issue. if you have `tried various things` then post them in your question. We can tell you where you are going the wrong.... On top of that I can do a simple google search and find the solution easily. Lack of research before asking a question usually gets you a lot of downvotes as well — John Ruddell, Nov 02 '15 at 22:07
Your exact question `How to write this result in a .txt output file ?` in a google search. [**google search**](https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=python+How+to+write+this+result+in+a+.txt+output+file)... [**first result**](http://stackoverflow.com/questions/5214578/python-print-string-to-text-file)... [**second result**](http://stackoverflow.com/questions/21963270/how-to-execute-a-python-script-and-write-output-to-txt-file). How hard is a google search? the answer is right there if you try. Remember google is your friend. Dont turn directly to SO. — John Ruddell, Nov 02 '15 at 22:12
@JohnRuddell Thanks for your answer. I thought that pasting my unsuccessful attempts with `json.dumps` would really pollute the question with noise, and reduce future readability. Well I won't add more, if nobody wants to answer, that's it, period. I thought it could be interesting. You say it's not, ok let it be then. — Basj, Nov 02 '15 at 22:13
dude, use a for loop on the dictionary and write for each new line. `for key, val in c.iteritems():` write the key with a comma and the value. you can write a string more than once. or use `writelines` for more. research writing to a file. — John Ruddell, Nov 02 '15 at 22:18
@JohnRuddell Thanks. I tried this, but then (with `for key, val in c.iteritems():`) it's no more sorted! Whereas, when I did print(c), it was sorted — Basj, Nov 02 '15 at 22:20
it wasn't ever sorted. dictionaries are not sorted and dont store any sorted arrangement. you can look into converting it into an `ordereddict` which will be sorted if you must — John Ruddell, Nov 02 '15 at 22:22
@JohnRuddell : In `print c`, it *was* sorted! (I tripled-checked) — Basj, Nov 02 '15 at 22:28
@Basj printing has nothing to do with its sort order. dictionaries are not sorted. there is no default sort order. I don't know how else to say this. If you dont believe me then look it up yourself. — John Ruddell, Nov 02 '15 at 22:34
@JohnRuddell : when doing `print c` in the 4th line (which is a `Counter`), the displayed result is sorted, period. (you can try). Where does this magical sort come from, I don't know, but it is sorted (Here I have hundreds of elements, and magically they are sorted with only the 4 lines of code I pasted.!)... I don't know where it comes from but it's true. — Basj, Nov 02 '15 at 22:36
a counter is a dictionary. yes its printing it in a sorted fashon that doesn't mean its sorted. look here for how to sort it http://stackoverflow.com/questions/9001509/how-can-i-sort-a-dictionary-by-key — John Ruddell, Nov 02 '15 at 22:42
@JohnRuddell Yes I agree with you about this. Maybe you have an idea on how to modify http://stackoverflow.com/a/33488137/1422096 to do the loop on dict sorted? I'm fighting with this since 20 minutes :) — Basj, Nov 02 '15 at 22:44
@Basj sure thing. I posted an answer. You are still asking people for way too much. a good programmer should be able to figure this stuff out. the resources are out there if you try to research it. — John Ruddell, Nov 02 '15 at 22:54

score 1 · Accepted Answer · answered Nov 02 '15 at 22:52

if you want to write it in a sorted manner you can do this.

from collections import Counter
wordlist = open('so.py', 'r').read().split()
word_counts = Counter(wordlist)

write_file = open('wordfreq.txt', 'w')
for w, c in sorted(word_counts.iteritems(), key=lambda x: x[1], reverse=True):
    write_file.write('{w}, {c}\n'.format(w=w, c=c))

score 0 · Answer 2 · answered Nov 02 '15 at 22:23

0

I think this may be the help you need: how to print the dictionary in the format you requested. The first four lines are your original code.

from collections import Counter
wordlist = open('so.py', 'r').read().split()
c = Counter(wordlist)
print c

outfile = open('output.txt', 'w')
for word, count in c.items():
    outline = word + ',' + str(count) + '\n'
    outfile.write(outline)

answered Nov 02 '15 at 22:23

Prune

76,765
14
60
81

Thanks! Just a small thing : how to have it sorted by frequency (`count`) ? In `print c`, it was magically sorted! (I tripled-checked) – Basj Nov 02 '15 at 22:27
The practical answer is that you search on line for "sort python dictionary by value". How did you not find something before you posted your comment? – Prune Nov 02 '15 at 22:53

score 0 · Answer 3 · answered May 02 '17 at 01:34

I think this can be done a little more simply. Also I used a context manager (with) to automatically close the files

from collections import Counter

with open('mybook.txt', 'r') as mybook:
    wordcounts = Counter(mybook.read().split())

with open('wordfreq.txt', 'w') as write_file:
    for item in word_counts.most_common():
        print('{}, {}'.format(*item), file=write_file)

If the file is particularly large, you can avoid reading it all into memory at once by using

    wordcounts = Counter(x for line in mybook for x in line.split())

Words, sorted by frequency, in a book (.txt file)

3 Answers3