7

Errors on Command Prompt

Basically, I have no idea why I'm getting this error.

Just to have more than an image, here is a similar message in code format. As it is more recent, the answer of this thread has already been mentioned in the message:

Preprocessing raw texts ...

---------------------------------------------------------------------------

LookupError                               Traceback (most recent call last)

<ipython-input-38-263240bbee7e> in <module>()
----> 1 main()

7 frames

<ipython-input-32-62fa346501e8> in main()
     32     data = data.fillna('')  # only the comments has NaN's
     33     rws = data.abstract
---> 34     sentences, token_lists, idx_in = preprocess(rws, samp_size=samp_size)
     35     # Define the topic model object
     36     #tm = Topic_Model(k = 10), method = TFIDF)

<ipython-input-31-f75213289788> in preprocess(docs, samp_size)
     25     for i, idx in enumerate(samp):
     26         sentence = preprocess_sent(docs[idx])
---> 27         token_list = preprocess_word(sentence)
     28         if token_list:
     29             idx_in.append(idx)

<ipython-input-29-eddacbfa6443> in preprocess_word(s)
    179     if not s:
    180         return None
--> 181     w_list = word_tokenize(s)
    182     w_list = f_punct(w_list)
    183     w_list = f_noun(w_list)

/usr/local/lib/python3.7/dist-packages/nltk/tokenize/__init__.py in word_tokenize(text, language, preserve_line)
    126     :type preserver_line: bool
    127     """
--> 128     sentences = [text] if preserve_line else sent_tokenize(text, language)
    129     return [token for sent in sentences
    130             for token in _treebank_word_tokenizer.tokenize(sent)]

/usr/local/lib/python3.7/dist-packages/nltk/tokenize/__init__.py in sent_tokenize(text, language)
     92     :param language: the model name in the Punkt corpus
     93     """
---> 94     tokenizer = load('tokenizers/punkt/{0}.pickle'.format(language))
     95     return tokenizer.tokenize(text)
     96 

/usr/local/lib/python3.7/dist-packages/nltk/data.py in load(resource_url, format, cache, verbose, logic_parser, fstruct_reader, encoding)
    832 
    833     # Load the resource.
--> 834     opened_resource = _open(resource_url)
    835 
    836     if format == 'raw':

/usr/local/lib/python3.7/dist-packages/nltk/data.py in _open(resource_url)
    950 
    951     if protocol is None or protocol.lower() == 'nltk':
--> 952         return find(path_, path + ['']).open()
    953     elif protocol.lower() == 'file':
    954         # urllib might not use mode='rb', so handle this one ourselves:

/usr/local/lib/python3.7/dist-packages/nltk/data.py in find(resource_name, paths)
    671     sep = '*' * 70
    672     resource_not_found = '\n%s\n%s\n%s\n' % (sep, msg, sep)
--> 673     raise LookupError(resource_not_found)
    674 
    675 

LookupError: 
**********************************************************************
  Resource punkt not found.
  Please use the NLTK Downloader to obtain the resource:

  >>> import nltk
  >>> nltk.download('punkt')
  
  Searched in:
    - '/root/nltk_data'
    - '/usr/share/nltk_data'
    - '/usr/local/share/nltk_data'
    - '/usr/lib/nltk_data'
    - '/usr/local/lib/nltk_data'
    - '/usr/nltk_data'
    - '/usr/lib/nltk_data'
    - ''
**********************************************************************
baduker
  • 19,152
  • 9
  • 33
  • 56
lesley2958
  • 2,538
  • 4
  • 15
  • 15

3 Answers3

15

Perform the following:

>>> import nltk
>>> nltk.download()

Then when you receive a window popup, select punkt under the identifier column which is locatedin the Module tab.

enter image description here

NealWalters
  • 17,197
  • 42
  • 141
  • 251
Leb
  • 15,483
  • 10
  • 56
  • 75
  • So I have tried this in the python shell, but it says "No module nltk," which I know I did install – lesley2958 Jun 13 '15 at 18:38
  • Did it say that after `import nltk`? – Leb Jun 13 '15 at 18:39
  • It did, but I figured out that nltk needed to be updated first, then the rest worked. Thank you! – lesley2958 Jun 13 '15 at 18:41
  • What about production environment? I mean, works fine locally, but what if I need to deploy to AWS using GitHub actions for CICD? Need to add something to the workflow file (.yml), or the Dockerfile maybe? – C. S. F. Junior Jan 12 '22 at 21:46
9

Do:

>>> import nltk
>>> nltk.download('punkt')
>>> from nltk import sent_tokenize

To download all dataset and models:

>>> nltk.download('all')

Ensure that you've the latest version of NLTK because it's always improving and constantly maintain:

$ pip install --upgrade nltk

Similar question on Windows/Linux but with the above code snippet don't help:

Similar question on Windows:

Similar question on Linux platform:

Similar question on OSX:

Similar question but has some authorization/authentication errors:

Similar question that tries to install NLTK data outside of python interpreter:

Community
  • 1
  • 1
alvas
  • 115,346
  • 109
  • 446
  • 738
2

To pre-install the punkt package with a single command line python -c 'import nltk; nltk.download("punkt")'.

jesugmz
  • 2,320
  • 2
  • 18
  • 33