1

I'm fine-tuning the BERT model in Named Entity Recognition task using json file dataset, but when I run that code always get an error:

    loss = model(b_input_ids, token_type_ids=None,
                 attention_mask=b_input_mask, labels=b_labels)
    # backward pass
    loss.backward()

I think the library has been changed but I don't know where can I find 'backward' attribute

امل بخش
  • 15
  • 2

1 Answers1

0

The TokenClassifierOutput class contains an attribute called loss, which has the type torch.FloatTensor, from which .backward() can be called.

token_classifier_output = model(b_input_ids,
                                token_type_ids=None,
                                attention_mask=b_input_mask,
                                labels=b_labels)
token_classifier_output.loss.backward()
kotatsuyaki
  • 1,441
  • 3
  • 10
  • 17
  • Thank you very much, it work with me, But after I fix this error I got another one with optimizer.step(). The error is: AttributeError: 'Adam' object has no attribute 'step' – امل بخش Dec 27 '22 at 13:41
  • To avoid extended discussion in the comments section, open a new question for your new problem(s) if needed. That aside, you seem to be running some legacy code (that was written for an older version of the `transformers` package) with a newer version of `transformers` installed, which requires you to rewrite some parts of the code. If running the code with the latest version of `transformers` is *not* required, a quicker way to fix it is to **1)** find out the version of `transformers` used before, and **2)** [install that specific version](https://stackoverflow.com/q/5226311/12122460) of it. – kotatsuyaki Dec 27 '22 at 17:00