open .txt file and place every word within a dictionary

Question

I wish to open a .txt file and enter all of the words within the file into a dictionary. After I would like to accumulate the total sum of words within the dictionary.

The .txt file contains 5 lines:

elephant calculator fish
towel onion fish
nandos pigeon tiger
cheeky peg lion
dog cat fish

this is what I currently have:

words = 0 
dictionary = []
with open('file.txt','r') as file:
    for x in inf:
        dictionary.split(x)
        words += 1
print(words)

Sorry about the awfully constructed question.

Answer 1

The simple way to get a count of unique words is to use a set. I put your text into a file called 'qdata.txt'.

The file is very small, so there's no need to read it line by line: just read the whole thing into a single string, then split that string on whitespace and pass the resulting list into the set constructor:

fname = 'qdata.txt'
with open(fname) as f:
    words = set(f.read().split())
print(words, len(words))

output

set(['towel', 'onion', 'nandos', 'calculator', 'pigeon', 'dog', 'cat', 'tiger', 'lion', 'cheeky', 'elephant', 'peg', 'fish']) 13

This works because "a set object is an unordered collection of distinct hashable objects". If you try to add a duplicate item to a set it's simply ignored. Please see the docs for further details.

---

For larger files, it is a good idea to read and process them line by line to avoid loading the whole file into RAM, but with modern OSes the file needs to be rather large before you see any benefit, due to file caching.

fname = 'qdata.txt'
words = set()
with open(fname) as f:
    for line in f:
        words.update(line.split())

print(words, len(words))

Answer 2

You have several problems but the basic strategy is sound

• dictionary is actually a list... which is what you want anyway. Rename it.
• you opened the file as file which is fine in Python 3 but frowned upon in Python 2 because it masks the builtin file object. People are still sensitive about that, so best use a different name.
• You didn't use the file variable, instead inventing something called inf.
• You split the wrong thing. You want to split the x line you read from the file.
• No need to count the words.... lists know how long they are.

So, this would work better

words = []
with open('file.txt') as fileobj:
    for x in fileobj:
        words += x.strip().split()
print(len(words))

collections.Counter is often used to count occurances of words. Assuming you can use anything in the standard lib, this would work (notice I lower cased so that Elephant and elephant count the same):

import collections
words = collections.Counter(int)
with open('file.txt') as fileobj:
    for x in fileobj:
        words.update(word.lower() for word in x.strip().split())
# words is a dict-like object with a count of each word
print(len(words))
print(words)
# lets pick one
print('elephant count', words['elephant'])

Answer 3

This may be inefficient, and never used in a case like this, but being as I am new as well, I wonder why the following wouldn't work for removing duplicates.

words = []
with open('file.txt') as fileobj:
    for x in fileobj:
        words += x.strip().split()
    for i in words:
        if words.count(i) > 1:
            words.remove(i)
print (len(words))
print (words)

Majority of code thanks to tdelaney.