1

what is EPOCH in neural network

I want EPOCH definition.

EPOCH is to update the weights.

So How does it work?

Change the "Training data(Input data)"?

Change the "Delta rule(Activation functions)"?

http://pages.cs.wisc.edu/~bolo/shipyard/neural/local.html

Bhargav Rao
  • 50,140
  • 28
  • 121
  • 140
user3669690
  • 83
  • 1
  • 1
  • 5
  • 1
    This post has some relevant info: http://stackoverflow.com/questions/4752626/epoch-vs-iteration-when-training-neural-networks – Aenimated1 May 15 '16 at 18:51

3 Answers3

10

This comes in the context of training a neural network with gradient descent. Since we usually train NNs using stochastic or mini-batch gradient descent, not all training data is used at each iterative step.

Stochastic and mini-batch gradient descent use a batch_size number of training examples at each iteration, so at some point you will have used all data to train and can start over from the beginning of the dataset.

Considering that, one epoch is one complete pass through the whole training set, means it is multiple iterations of gradient descent updates until you show all the data to the NN, and then start again.

Dr. Snoopy
  • 55,122
  • 7
  • 121
  • 140
  • 1
    What does one complete pass do? why does more than one complete pass necessary if you are using the exact same training data? I thought one pass will find the minimum error from the gradient descent. – mskw Oct 09 '17 at 20:08
  • @mskw Gradient descent is an iterative algorithm, it does not find the minimum in a single iteration. – Dr. Snoopy Oct 09 '17 at 20:37
  • So does epoch has anything to do with gradient descent? – mskw Oct 09 '17 at 20:54
  • @mskw Yes, that is what my answer says. Batches come from the need to use only part of the training set at each iteration, in the common case that you might not fit the whole dataset in memory or doing an iteration with the full dataset is too computationally expensive. An epoch is then just a full pass over the training set. – Dr. Snoopy Oct 09 '17 at 20:58
2

To put it really simple:

Epoch is a function in which everything happens. Within one epoch, you start forward propagation and back propagation. Within one epoch you make neuron activate, calculate loss, get partial derivatives of loss function and you update new values with your weights. And when all these is done, you start new epoch, and then new one etc. The number of epochs is not really important. What matter is the change of your loss function, derivatives etc. So when you are happy with results, you can stop with epoch iteration and get your model out :)

filtertips
  • 801
  • 3
  • 12
  • 25
0

Epoches is single pass through whole training dataset. Traditional Gradient Descent computes the gradient of the loss function with regards to parameters for the entire training data set for a given number of epochs.

Naren Babu R
  • 453
  • 2
  • 9
  • 33
  • 1
    A single pass is visiting all the neurons from the first layer, then hidden layer and then the last/output layer. – Naren Babu R Oct 10 '17 at 03:15