5

I have finetuned the SciBERT model on the SciIE dataset. The repository uses AllenNLP to finetune the model. The training is executed as follows:

python -m allennlp.run train $CONFIG_FILE  --include-package scibert -s "$@"

After a successful training I have a model.tar.gz file as an output that contains weights.th, config.json, and vocabulary folder. I have tried to load it in the allenlp predictor:

from allennlp.predictors.predictor import Predictor
predictor = Predictor.from_path("model.tar.gz")

But I get the following error:

ConfigurationError: bert-pretrained not in acceptable choices for dataset_reader.token_indexers.bert.type: ['single_id', 'characters', 'elmo_characters', 'spacy', 'pretrained_transformer', 'pretrained_transformer_mismatched']. You should either use the --include-package flag to make sure the correct module is loaded, or use a fully qualified class name in your config file like {"model": "my_module.models.MyModel"} to have it imported automatically.

I have never worked with allenNLP, so I am quite lost about what to do.

For reference, this is the part of the config that describer token indexers

"token_indexers": {
            "bert": {
                "type": "bert-pretrained",
                "do_lowercase": "false",
                "pretrained_model": "/home/tomaz/neo4j/scibert/model/vocab.txt",
                "use_starting_offsets": true
            }
        }

I am using allenlp version

Name: allennlp Version: 1.2.1

Edit:

I think I have made a lot of progress, I have to use the same version that was used to train the model and I can import the modules like so:

from allennlp.predictors.predictor import Predictor
from scibert.models.bert_crf_tagger import *
from scibert.models.bert_text_classifier import *
from scibert.models.dummy_seq2seq import *
from scibert.dataset_readers.classification_dataset_reader import *

predictor = Predictor.from_path("scibert_ner/model.tar.gz")
dataset_reader="classification_dataset_reader")
predictor.predict(
  sentence="Did Uriah honestly think he could beat The Legend of Zelda in under three hours?"
)

Now I get an error:

No default predictor for model type bert_crf_tagger.\nPlease specify a predictor explicitly

I know that I can use the predictor_name to specify a predictor explicitly, but I haven't got the faintest idea which name to pick that would work

Tomaž Bratanič
  • 6,319
  • 2
  • 18
  • 31
  • What repository is this from? – petew Nov 17 '20 at 18:37
  • its from the official scibert: https://github.com/allenai/scibert – Tomaž Bratanič Nov 17 '20 at 19:55
  • And the training script is available under /scripts/train_allennlp_local.sh – Tomaž Bratanič Nov 17 '20 at 20:12
  • Feel like I need to include the model somehow that is available at: scibert/models, not exactly sure how to load it in python... is there an example of CLI predict command and the data structure? Very hard to find anything? – Tomaž Bratanič Nov 18 '20 at 20:04
  • It looks to me like this repo uses an old version of AllenNLP: https://github.com/allenai/scibert/blob/master/requirements.txt#L1. There is no "bert-pretrained" token indexer in the AllenNLP library, but maybe there used to be? I don't know where else that would come from. – petew Nov 18 '20 at 20:50
  • Ok, it seems it is trained on 0.9.0... I have another issue that is bert_crf_tagger is not a registered name for Model Help me fix this and I gladly give you 250 + 25 points :) – Tomaž Bratanič Nov 18 '20 at 21:22
  • and that model is available under scibert/ folder, just no clue how to import it? – Tomaž Bratanič Nov 18 '20 at 21:22
  • I have made some edits – Tomaž Bratanič Nov 19 '20 at 12:00
  • 2
    There's [someone with that problem](https://github.com/allenai/scibert/issues/102#issue-673470385). Adapting from [this answer](https://github.com/allenai/allennlp/issues/2849#issuecomment-493166329), would say something like «If the model type known then AllenNLP can choose a predictor automatically, but there's no registered predictor of type `bert_crf_tagger` so you'll need to supply one yourself. [Read here for Creating a Predictor](https://docs.allennlp.org/v1.0.0rc4/tutorials/getting_started/predicting_paper_venues/predicting_paper_venues_pt2/#creating-a-predictor).» – Tiago Martins Peres Nov 19 '20 at 14:44

1 Answers1

3

I have seen a lot of people having this problem. Upon going through the repository code, I found this to be the easiest way to run the predictions:

python -m allennlp.run predict /path/to/saved_model/model.tar.gz /path/to/test.txt\
  --include-package scibert --use-dataset-reader\
  --output-file /path/to/where/you/want/predict.txt\
  --predictor  sentence-tagger --batch-size 16

What did I add? The predictor sentence-tagger. Once you go through the repository, you would find that the registered predictor is sentence-tagger. Although the DEFAUL_DICT of the taggers contain sentence_tagger. A lot of confusion, right? Tell me!

This answer also saves you from writing a predictor.

paradocslover
  • 2,932
  • 3
  • 18
  • 44
  • can you point to the referenced code? depends on which task you want to do, probably ner classification differs from text classification predictor – Tomaž Bratanič Dec 10 '20 at 10:31
  • I didn't quite get your question ... But I guess you want me to ask where exactly did I find "sentence-tagger". To answer that, I found it here https://github.com/ibeltagy/allennlp/blob/master/allennlp/predictors/sentence_tagger.py. The predictor for "sentence_tagger" is registered as @Predictor.register('sentence-tagger') – paradocslover Dec 11 '20 at 02:53