0

I have wdict as a dictionary and i want to add all unique words to it which are scanned from the text files stored at path and converted to list of words by CleanDoc(). I an getting error AttributeError: 'dict' object has no attribute 'union'. What should i do?

import collections
import os.path
import glob
import nltk

wdict = {}
path = "C://Python27//Corpus Files//*.*"


#this function cleans up a doc (removes stopwords etc)
def cleanDoc(doc):
    stopset = set(nltk.corpus.stopwords.words('english'))
    stemmer = nltk.PorterStemmer()
    tokens = nltk.WordPunctTokenizer().tokenize(doc)
    clean = [token.lower() for token in tokens if token.lower() not in stopset and len(token) > 3 and token.isalpha() and not 'reuter']
    final = [stemmer.stem(word) for word in clean]
    return final

for text in glob.glob(path):
    f = open(text)
    data= f.read()
    words = cleanDoc(data)
    wdict = wdict.union(words)
    print wdict
DummyGuy
  • 425
  • 1
  • 8
  • 20
  • 4
    Take a look at the [dict API](http://docs.python.org/2/library/stdtypes.html#mapping-types-dict). There is no `union` method for a `dict`. – SethMMorton Feb 08 '14 at 21:01
  • 1
    Note that (once you make it a `set`) you will get an `UnboundLocalError` from trying to assign to a `global` without declaring it as such. – roippi Feb 08 '14 at 21:06

3 Answers3

3

You can use a set instead of a dict:

wdict = set() # `wset` will make a better name

Also probably wdict.update(words) will look better than wdict = wdict.union(words)

Nigel Tufnel
  • 11,146
  • 4
  • 35
  • 31
  • 1
    To be clear, there is no "empty set literal" in python, so the only way to make an empty set is with `set()`. – SethMMorton Feb 08 '14 at 21:04
  • @Nigel Tufnel: `wdict.update(words)` returns `None` so you can't assign it to a variable, whereas `wdict.union(words)` returns the updated set. – superjump Feb 08 '14 at 21:07
  • @superjump Yup, so that's why they left off the `wdict =` in front of the `update` call. – SethMMorton Feb 08 '14 at 21:08
  • If you change `wdict` from a dict to a set, best to change its name too, having a variable `wdict` which is not actually a dict could be confusing. – glyphobet Feb 08 '14 at 21:20
2

Python Dictionary object does not have a union method. As suggested in the error. Union method is only available over Sets.

You should look at the SO answer :- How to merge two Python dictionaries in a single expression?

My favorite is :-



       w_dicts.update(words)

But this is purely because of personal choice.

Hope this helps.

Community
  • 1
  • 1
ayushmad
  • 619
  • 1
  • 6
  • 9
  • Notice that they are actually using sets, they thought that `{}` created an emtpy set, not an empty dict. That's really the error here. – SethMMorton Feb 08 '14 at 21:09
0

Change wdict = {} to wdict = set().

superjump
  • 151
  • 4