0

So, I just learned Pytorch and they say that you must put the NN in train mode by .train() method, then while inferencing .eval() mode. I'm reading this tutorial and there's no .train() at all. Why is that?

https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#sphx-glr-beginner-blitz-cifar10-tutorial-py

Alex Deft
  • 2,531
  • 1
  • 19
  • 34
  • Possible duplicate of [What does model.train() do in pytorch?](https://stackoverflow.com/questions/51433378/what-does-model-train-do-in-pytorch) – Berriel Jul 07 '19 at 12:18
  • 2
    Please, do not post the same question **at the same time** on SO and [Discuss PyTorch](https://discuss.pytorch.org/t/where-theres-no-train-method-in-this-pytorch-official-tutorial/49892). At least, wait for _one day_ for an answer in one of them. – Berriel Jul 07 '19 at 12:19

1 Answers1

3

.train() sets the self.training property of the module to True. .eval() sets it to False.

As can be seen in the source for nn.Module, this property is initially set to True. So, unless you have called eval() before you start training, you do not necessarily (as per the current implementation) need to call train(). But the important point is that the module should be in the self.training=True state at the time of training, so it is probably a good practice to do so anyway.

Also, currently, only some modules (such as dropout and batchnorm) change their behavior based on the self.training property. So you don't necessarily have to call .train() and .eval() if you are not using those specific modules, but again, it is probably good practice to do so anyway for future-proofing your code.

adeelh
  • 527
  • 5
  • 10