3

The following code fails for me when I do not have a working internet connection, with the error message that I have detailed below. I have tried a lot of things including configuring and checking the .flair folder and the .cache folder that the transformers package uses. I also checked that distilbert-base-uncased is a current model identifier on https://huggingface.co/models as the error message suggested. What can I do to fix this? Details below.

Code that fails

from flair.models import TextClassifier
en_classifier = TextClassifier.load('en-sentiment')

Error:

Can't load tokenizer for 'distilbert-base-uncased'. Make sure that:

- 'distilbert-base-uncased' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'distilbert-base-uncased' is the correct path to a directory containing relevant tokenizer files

Additional Details:

Flair version: 0.8.0.post1'

Environment variables needed because my app needs to run standalone with no internet connection:

TRANSFORMERS_OFFLINE=1
HF_DATASETS_OFFLINE=1

I have the model files located in the .flair directory that is configured by flair.cache_root. The code that works for me when I load other models is:

from flair.models import TextClassifier
rnn_classifier = TextClassifier.load('sentiment-fast')

even this works:

from flair.models import SequenceTagger
tagger = SequenceTagger.load('ner')

.flair directory has:

  • sentiment-en-mix-distillbert_4.pt
  • sentiment-en-mix-ft-rnn.pt
  • ner-english (folder)
  • pos-english (folder)

~/.cache/huggingface/transformers directory has:

  • 0e1bbfda7f63a99bb52e3915dcf10c3c92122b827d92eb2d34ce94ee79ba486c.d789d64ebfe299b0e416afc4a169632f903f693095b4629a7ea271d5a0cf2c99
  • 0e1bbfda7f63a99bb52e3915dcf10c3c92122b827d92eb2d34ce94ee79ba486c.d789d64ebfe299b0e416afc4a169632f903f693095b4629a7ea271d5a0cf2c99.json
  • 75abb59d7a06f4f640158a9bfcde005264e59e8d566781ab1415b139d2e4c603.7f2721073f19841be16f41b0a70b600ca6b880c8f3df6f3535cbc704371bdfa4
  • 75abb59d7a06f4f640158a9bfcde005264e59e8d566781ab1415b139d2e4c603.7f2721073f19841be16f41b0a70b600ca6b880c8f3df6f3535cbc704371bdfa4.json
  • 8c8624b8ac8aa99c60c912161f8332de003484428c47906d7ff7eb7f73eecdbb.20430bd8e10ef77a7d2977accefe796051e01bc2fc4aa146bc862997a1a15e79
  • 8c8624b8ac8aa99c60c912161f8332de003484428c47906d7ff7eb7f73eecdbb.20430bd8e10ef77a7d2977accefe796051e01bc2fc4aa146bc862997a1a15e79.json
Rahul P
  • 2,493
  • 2
  • 17
  • 31

0 Answers0