How to load a word list into Python

Question

I'm working through an introductory Python programming course on MIT OCW. On this problem set I've been given some code to work on and a text file. The code and the text file are in the same folder. The code looks like this:

import random 
import string

def load_words( ): 
       print "Loading word list from file..." 
       inFile = open (WORDLIST_FILENAME, 'r', 0) 
       line = inFile.readline( ) 
       wordlist = string.split (line) 
       print " ", len(wordlist), "words loaded." 
       return wordlist

def choose_word (wordlist): 
       return random.choice (wordlist)

wordlist = load_words ( )

When I run the code as it is, the problem set instructions say I should get this:

Loading word list from file... 
55900 words loaded.

For some reason though, when I run the code I get:

Loading word list from file... 
1 words loaded

I've tried omitting the 2nd and 3rd parameters from the input to the open function but to no avail. What could the problem be?

Moreover, when I try to print the value of wordlist I get

['AA']

When I print the value of line within the context of the relevant function I get:

AA

The text file does begin with 'AA', but what about all of the letters that follow?

related to http://stackoverflow.com/questions/11555468/how-should-i-read-a-file-line-by-line-in-python. Definately look there for a good example on how to read in multiple lines simply and concisely. — Ross, Apr 16 '15 at 05:55
@AshokaLella the word list is a text file called words containing a massive list of apparently random letters — user224530, Apr 16 '15 at 06:09
@user224530, I understand that it is a collection of random letters. Do you have a single word per line or multiple words in each line? Can you post first few lines of the wordlist? — Ashoka Lella, Apr 16 '15 at 06:11
Thanks Ashoka for your help. The word list starts like this: AAAAHAAHEDAAHINGAAHSAALAA... and it goes on and on... I don't see how meaningful words can be constructed out of this — user224530, Apr 16 '15 at 06:33

Ashoka Lella · Answer 1 · 2015-04-16T05:46:14.713

2

line = inFile.readline( ) should be readlines(), plural.
readline would read only a single line. The reason why only one word is read.
Using readlines() would give you a list delimited by new line characters in your input file.

edited Apr 16 '15 at 05:46

answered Apr 16 '15 at 05:39

Ashoka Lella

6,631
1
30
39

I just tried that, but now it gives me an error saying: AttributeError: 'list' object has no attribute 'split' – user224530 Apr 16 '15 at 06:06
@user224530, you cant run a split on list. you may want to run a split on individual elements in a list. I cant help you further unless you post a sample of your wordlist file. If you have a single word on each line `wordlist=inFile.readlines()` should work. Else you'll have to modify your logic – Ashoka Lella Apr 16 '15 at 06:09

score 1 · Answer 2 · answered Apr 16 '15 at 06:16

raw file like this:

cat wordlist.txt                                                                                                                                     
aa
bb
cc
dd
ee

python file like this:

import random

def load_words(WORDLIST_FILENAME):
       print "Loading word list from file..."
       wordlist = list()
       # 'with' can automate finish 'open' and 'close' file
       with open(WORDLIST_FILENAME) as f:
            # fetch one line each time, include '\n'
            for line in f:
                # strip '\n', then append it to wordlist
                wordlist.append(line.rstrip('\n'))
       print " ", len(wordlist), "words loaded."
       print '\n'.join(wordlist)
       return wordlist

def choose_word (wordlist):
       return random.choice (wordlist)

wordlist = load_words('wordlist.txt')

then result:

python load_words.py                                                                                                                                 
Loading word list from file...
  5 words loaded.
aa
bb
cc
dd
ee

score 1 · Answer 3 · answered Apr 16 '15 at 07:09

the function u have written can read words in a single line. It assumes all words are written in single line in text file and hence reads that line and creates a list by splitting it. However, it appears your text file contains some newlines also. Hence u can replace the following with:

line = inFile.readline( ) 
wordlist = string.split (line)

with:

wordlist =[] 
for line in inFile:
    line = line.split()
    wordlist.extend(line)
print " ", len(wordlist), "words loaded."

How to load a word list into Python

3 Answers3

Linked