27

Official comment shows that "This has any effect only on modules such as Dropout or BatchNorm." But I don't understand its implementation.

闫伯元
  • 445
  • 1
  • 4
  • 9
  • 6
    it's simple and straightforward, dropout and batchnorm are disabled in the evaluation mode if you have such steps in your model. – Wasi Ahmad Jan 08 '18 at 09:46
  • 1
    Does this answer your question? [What does model.eval() do in pytorch?](https://stackoverflow.com/questions/60018578/what-does-model-eval-do-in-pytorch) – iacob Mar 08 '21 at 17:49

1 Answers1

27

Dropout and BatchNorm (and maybe some custom modules) behave differently during training and evaluation. You must let the model know when to switch to eval mode by calling .eval() on the model.

This sets self.training to False for every module in the model. If you are implementing your own module that must behave differently during training and evaluation, you can check the value of self.training while doing so.

litesaber
  • 401
  • 3
  • 2
  • 3
    In the official PyTorch tutorial (60 Minute Blitz, Training A Classifier), they did not use `.eval()` when switching to evaluation (the test set). Maybe they didn't use it because there were no Dropout or Batchnorm layers. But I think from an educational standpoint they should have used it anyway (even if it would have no effect) or at least mentioned it since it's required in the general case =/ – xjcl Mar 11 '19 at 22:11
  • PyTorch code mentions `self.train(False)`, and not `self.training` like it is written in the previous answer, cf. https://pytorch.org/docs/stable/_modules/torch/nn/modules/module.html#Module.eval (perhaps a recent change though). @xjcl as PyTorch documentation says, "This has any effect only on certain modules.", so I guess you can systematically use it without trouble, cf. https://pytorch.org/docs/stable/nn.html#torch.nn.Module.eval – Blupon Jan 22 '20 at 17:10