3

I have an AI project, which uses a Backpropagation neural network.

It is training for about 1 hour, and it has trained 60-70 inputs from all 100 inputs. I mean, 60-70 inputs are correct in the condition of Backpropagation. (the number of trained inputs is moving between 60 and 70).

And currently, more than 10000 epochs are completed, and each epoch is taking almost 0.5 seconds.

How to know if the neural network can be trained successfully if I leave it for a long time? (or it can't train better?)

Mahdi Ghiasi
  • 14,873
  • 19
  • 71
  • 119

1 Answers1

8

Check out my answer to this question: whats is the difference between train, validation and test set, in neural networks?

You should use 3 sets of data:

  • Training
  • Validation
  • Testing

The Validation data set tells you when you should stop (as I said in the other answer):

The validation data set is used to minimize overfitting. You're not adjusting the weights of the network with this data set, you're just verifying that any increase in accuracy over the training data set actually yields an increase in accuracy over a data set that has not been shown to the network before, or at least the network hasn't trained on it (i.e. validation data set). If the accuracy over the training data set increases, but the accuracy over then validation data set stays the same or decreases, then you're overfitting your neural network and you should stop training.

A good method for validation is to use 10-fold (k-fold) cross-validation. Additionally, there are specific "strategies" for splitting your data set into training, validation and testing. It's somewhat of a science in itself, so you should read up on that too.

Update

Regarding your comment on the error, I would point you to some resources which can give you a better understanding of neural networks (it's kinda math heavy, but see below for more info):

  1. http://www.colinfahey.com/neural_network_with_back_propagation_learning/neural_network_with_back_propagation_learning_en.html
  2. http://www.willamette.edu/~gorr/classes/cs449/linear2.html

Section 5.9 of Colin Fahey article describes it best:

Backward error propagation formula:
The error values at the neural network outputs are computed using the following formula: 

Error = (Output - Desired);  // Derived from: Output = Desired + Error;

The error accumulation in a neuron body is adjusted according to the output of the neuron body and the output error (specified by links connected to the neuron body). Each output error value contributes to the error accumulator in the following manner:

ErrorAccumulator += Output * (1 - Output) * OutputError;
Community
  • 1
  • 1
Kiril
  • 39,672
  • 31
  • 167
  • 226
  • Thanks, but I have a question : What is training set error and validation set error values? How to calculate them? – Mahdi Ghiasi Jan 19 '12 at 17:23
  • @Mahdi, are you using a neural network library? If you are, then the library automatically should calculate the errors while it's training your neural network. However, I've updated the question above with more info and details on how the error is usually computed. – Kiril Jan 19 '12 at 18:02
  • I'm asking about the error function that usually shown in charts and it must go to 0 after epoches. The Y-axis of this image : http://www.cs.cornell.edu/boom/2004sp/projectarch/appofneuralnetworkcrystallography/images/NeuralNetworkErrorGraph.jpg .What is it and how to calculate it? – Mahdi Ghiasi Jan 19 '12 at 18:18
  • As I stated in my update, the error shown in that graph is the difference between the output neuron and the actual value you're trying to estimate from the training instance. You may never get the error to go down to 0, as a matter of fact in most practical applications it **never** goes down to 0. If you're using a neural network library, then all of those calculations are already done for you. – Kiril Jan 19 '12 at 18:56
  • difference between the output neuron and the actual value of which input? or sum of the differences between output neuron and actual value? or average of them??? – Mahdi Ghiasi Jan 19 '12 at 19:08
  • See section 5.9 of this article: http://www.colinfahey.com/neural_network_with_back_propagation_learning/neural_network_with_back_propagation_learning_en.html The error is literally `Error = (Output - Desired)`. – Kiril Jan 19 '12 at 19:29
  • I think the answer of my question is section 6.3 of Colinfahey: 6.3 Overall training error. – Mahdi Ghiasi Jan 19 '12 at 21:02
  • @Mahdi, ok, got it... that's a different kind of error. What you showed me in the link to the image looked like the output error. – Kiril Jan 19 '12 at 21:41
  • When should I stop training? when the current overall validation error is bigger than the previous one? or average of 10-20 of them? or ...? – Mahdi Ghiasi Jan 20 '12 at 14:12
  • @Mahdi, you should stop training when your validation accuracy stops increasing. Your training accuracy might increase, but if your validation accuracy stops increasing, then any more training will result in over-fitting. – Kiril Jan 20 '12 at 15:52