spaCy can't load model ONLY when calling "rasa train"

Question

I'm training a rasa model via command line but spaCy seems to be unable to load my language model pt_core_news_sm only when I try to train via terminal.

Everything is done inside my venv and executed as admin;

A may load the model when calling spaCy from python:

import spacy
spacy.load("pt_core_news_sm")

Training via rasa_nlu in a python script works as well.

But when I call rasa train I get this traceback:

2020-02-06 02:03:19 INFO     rasa.nlu.utils.spacy_utils  - Trying to load spacy model with name 'pt_core_news_sm'
Traceback (most recent call last):
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\utils\spacy_utils.py", line 51, in load_model
    return spacy.load(spacy_model_name, disable=["parser"])
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\spacy\__init__.py", line 30, in load
    return util.load_model(name, **overrides)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\spacy\util.py", line 169, in load_model
    raise IOError(Errors.E050.format(name=name))
OSError: [E050] Can't find model 'pt_core_news_sm'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\aliss\AppData\Local\Programs\Python\Python36\Scripts\rasa.exe\__main__.py", line 9, in <module>
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\__main__.py", line 76, in main
    cmdline_arguments.func(cmdline_arguments)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\cli\train.py", line 76, in train
    additional_arguments=extract_additional_arguments(args),
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\train.py", line 50, in train
    additional_arguments=additional_arguments,
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\asyncio\base_events.py", line 473, in run_until_complete
    return future.result()
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\train.py", line 101, in train_async
    additional_arguments,
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\train.py", line 188, in _train_async_internal
    additional_arguments=additional_arguments,
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\train.py", line 245, in _do_training
    persist_nlu_training_data=persist_nlu_training_data,
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\train.py", line 474, in _train_nlu_with_validated_data
    persist_nlu_training_data=persist_nlu_training_data,
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\train.py", line 74, in train
    trainer = Trainer(nlu_config, component_builder)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\model.py", line 147, in __init__
    self.pipeline = self._build_pipeline(cfg, component_builder)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\model.py", line 159, in _build_pipeline
    component = component_builder.create_component(component_cfg, cfg)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\components.py", line 515, in create_component
    component = registry.create_component_by_config(component_config, cfg)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\registry.py", line 228, in create_component_by_config
    return component_class.create(component_config, config)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\utils\spacy_utils.py", line 81, in create
    nlp = cls.load_model(spacy_model_name)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\utils\spacy_utils.py", line 58, in load_model
    "en_core_web_md en".format(spacy_model_name)
rasa.nlu.model.InvalidModelError: Model 'pt_core_news_sm' is not a linked spaCy model.  Please download and/or link a spaCy model, e.g. by running:
python -m spacy download en_core_web_md
python -m spacy link en_core_web_md en

I've tried the following download strategies with python and python3. The first seemed to be the answer for this question, but didn't work as well:

pip install https://github.com/explosion/spacy-models/releases/download/pt_core_news_sm-2.2.5/pt_core_news_sm-2.2.5.tar.gz

python -m spacy download pt

python -m spacy download pt_core_news_sm

This is my configuration file:

language: "pt_core_news_sm"

pipeline:
- name: "SpacyNLP"                 
- name: "tokenizer_spacy"          
- name: "ner_crf"                
- name: "intent_featurizer_spacy"     
- name: "intent_classifier_sklearn"  
- name: "ner_synonyms"               

policies:
  - name: "FormPolicy"

I'm using...

spacy==2.2.3

rasa==1.7.0
rasa-core==0.14.5
rasa-core-sdk==0.14.0
rasa-nlu==0.15.0
rasa-sdk==1.7.0

Are you absolutely positively sure that you are trying this inside the same python environment? Also, did you update your spacy version recently, or are you working with several different environments? — dennlinger, Feb 06 '20 at 12:02
Yes, I've updated, but, as I said, training inside python via rasa_nlu worked well. I found that, in command line training, rasa searchs for the language model outside my virtualenv, even if I call it from inside. Downloading my language model outside the virtual environment solved it. — Alisson Correa, Feb 06 '20 at 15:03

Alisson Correa · Answer 1 · 2020-02-06T15:08:57.070

2

Solved it. Apparently, calling rasa train from inside venv searchs for the language model outside my virtualenv. I had to download the model executing python outside venv and it completed the training. Even so, I don't know why this happens.

edited Feb 06 '20 at 15:08

answered Feb 06 '20 at 15:01

Alisson Correa

357
2
8

score 0 · Answer 2 · answered Jan 19 '21 at 13:06

For Windows users --

Please check your Python Environment first. Settings --> Project Interpreter. If it's Anaconda virtual environment.

Open CMD as admin(right-click cmd and choose run as administrator)

Activate your environment. e.g:

activate rasaTesting(Name of your project/env)

Create shortcut link for spacy model like (Download spacy model if haven't already)-

python -m spacy link en_core_web_md en

Attached image of env.

spaCy can't load model ONLY when calling "rasa train"

2 Answers2