ValueError: The model did not return a loss from the inputs, only the following keys: logits

Question

I want to make a metaphor detector model. As a pretrained model, I use a DistilBert model that I have, previously, fine-tuned with masking (this is the model I use to make the new metaphor detection model). The new model is the one that gives me an error (the one in the code). I try to do sequence classification. But, the error arises when training it.

An example of the dataset is: 001 T1 M darkness is light's elder brother. 150 T2 N the knight rode the wild horse. It has 150 lines and the M/N is the label and the fourth column the sentence.

dataset = load_dataset('text', data_files='/content/drive/MyDrive/project/metaphors_dataset.txt')

# 90% train, 10% test + validation
train_testvalid = dataset["train"].train_test_split(test_size=0.1)
# Split the 10% test + valid in half test, half valid
test_valid = train_testvalid['test'].train_test_split(test_size=0.5)
# gather everyone if you want to have a single DatasetDict
train_test_valid_dataset = DatasetDict({
    'train': train_testvalid['train'],
    'test': test_valid['test'],
    'valid': test_valid['train']})

print(train_test_valid_dataset)

tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")

def preprocess_function(examples):
   return tokenizer(examples["text"], truncation=True)
 
tokenized_train = train_test_valid_dataset["train"].map(preprocess_function, batched=True)
tokenized_test = train_test_valid_dataset["test"].map(preprocess_function, batched=True)

path_model = "/content/drive/MyDrive/project/my_model.pt"
model = AutoModelForSequenceClassification.from_pretrained(path_model, num_labels=2, from_tf=True) 

def compute_metrics(pred):
  labels = pred.label_ids
  preds = pred.predictions.argmax(-1)
  accuracy = accuracy_score(labels, preds)
  f1 = f1_score(labels, preds, average='weighted')
  return {"accuracy": accuracy, "f1": f1}

training_args = TrainingArguments(
    output_dir=repo_name,
    evaluation_strategy='epoch',
    learning_rate=2e-5,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    num_train_epochs=2,
    weight_decay=0.01,
    save_total_limit=1,
    save_strategy='epoch',
    load_best_model_at_end=True,
    metric_for_best_model='f1',
    greater_is_better=True,
    adam_epsilon=1e-8,
    adam_beta2=0.01
)
trainer = Trainer(
   model=model,
   args=training_args,
   train_dataset=tokenized_train,
   eval_dataset=tokenized_test,
   tokenizer=tokenizer,
   data_collator=data_collator,
   compute_metrics=compute_metrics,
)

The error is in this line:

trainer.train()

Error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-97-3435b262f1ae> in <module>
----> 1 trainer.train()

3 frames
/usr/local/lib/python3.9/dist-packages/transformers/trainer.py in train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
   1631             self._inner_training_loop, self._train_batch_size, args.auto_find_batch_size
   1632         )
-> 1633         return inner_training_loop(
   1634             args=args,
   1635             resume_from_checkpoint=resume_from_checkpoint,

/usr/local/lib/python3.9/dist-packages/transformers/trainer.py in _inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)
   1900                         tr_loss_step = self.training_step(model, inputs)
   1901                 else:
-> 1902                     tr_loss_step = self.training_step(model, inputs)
   1903 
   1904                 if (

/usr/local/lib/python3.9/dist-packages/transformers/trainer.py in training_step(self, model, inputs)
   2643 
   2644         with self.compute_loss_context_manager():
-> 2645             loss = self.compute_loss(model, inputs)
   2646 
   2647         if self.args.n_gpu > 1:

/usr/local/lib/python3.9/dist-packages/transformers/trainer.py in compute_loss(self, model, inputs, return_outputs)
   2688         else:
   2689             if isinstance(outputs, dict) and "loss" not in outputs:
-> 2690                 raise ValueError(
   2691                     "The model did not return a loss from the inputs, only the following keys: "
   2692                     f"{','.join(outputs.keys())}. For reference, the inputs it received are {','.join(inputs.keys())}."

ValueError: The model did not return a loss from the inputs, only the following keys: logits. For reference, the inputs it received are input_ids,attention_mask.

Thank you!

I want it to start training and then to evaluate it.

Demex Khal · Answer 1 · 2023-06-05T08:51:17.753

0

The data-set must have columns with names "text" and "labels". 1

edited Jun 05 '23 at 08:51

answered Jun 05 '23 at 08:49

Demex Khal

1
1

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Jun 05 '23 at 10:05
This does not provide an answer to the question. Once you have sufficient [reputation](https://stackoverflow.com/help/whats-reputation) you will be able to [comment on any post](https://stackoverflow.com/help/privileges/comment); instead, [provide answers that don't require clarification from the asker](https://meta.stackexchange.com/questions/214173/why-do-i-need-50-reputation-to-comment-what-can-i-do-instead). - [From Review](/review/late-answers/34499179) – dpapadopoulos Jun 07 '23 at 15:14

ValueError: The model did not return a loss from the inputs, only the following keys: logits

1 Answers1