I want to write a paper about convolutional neural network, the accuracy of validation dataset of my cnn model is fluctuating from 98 to 99, the accuracy of test dataset is not the same every time. So i am not sure which time I should write the accuracy, the total number of epochs in the experiment is 6000 times. Can I choose the highest accuracy of 10 epochs in the last epoch as the accuracy of the paper, or how should i do it? Thank you very much!
Asked
Active
Viewed 616 times
2 Answers
0
10 measurements isn't enough for deriving statical attributes with significant property... you can list all 10 or give the low, high, avg and median, those should be enough to contract your network acc and compare it to others networks
shahaf
- 4,750
- 2
- 29
- 32
-
But to compare with other papers, few papers will list the lowest, the highest, the mean and the median, usually only one number, so how should such a statement be compared, thank you. – ohdoughnut May 01 '18 at 12:51
-
@ohdoughnut most papers post the best value, to make them look good, by posting the statistical attributes of your acc, it can be compared to any other paper, e.g at the worst case my network perform better... – shahaf May 01 '18 at 14:18
0
I have the same problem. These are the approaches I have been using:
Periodically reduce the learning rate
Periodically reducing the learning rate not only increases validation accuracy, it also reduces the variance of the accuracy at the end of training.
For example, dividing the learning rate by 5
Epochs Learning rate
0-4000 5e-4
4000-5000 1e-4
5000-6000 2e-5
etc
You can also monitor this automatically. For example, if the loss has not decreased significantly for the last N batches, decrease the learning rate and restart the count. After M such decreases, stop training.
Repeat the experiment
The accuracy you get will be dependant on the initial random weights assigned when the network is created. Thus even when using learning rate reduction, you will still get different results. Repeat the experiment and take the average. This make take a long time, depending on your dataset.
geometrikal
- 3,195
- 2
- 29
- 40
-
At first, thank you very much. I have tried to change the learning rate of 0.0001 to 0.00005 and 0.00001, but the final accuracy is no learning rate of 0.0001, so the learning rate can only be 0.0001. and I see in an paper"In order to determine convergence, training ended when there was no improvement on the validation set within ten consecutive epochs. The epoch with the highest validation score was chosen. "in this paper, the highest accuracy is chosen. Is the accuracy chosen by the individual himself? – ohdoughnut May 01 '18 at 12:46
-
@ohdoughnut I've updated what I mean by periodically decrease the learning rate. Which paper are you referring to? Sounds like the individual chose the best learning rate from 10 options, I dont really like this approach. – geometrikal May 01 '18 at 13:07
-
This paper is "CNN-based Sensor Fusion Techniques for Multimodal Human Activity Recognition". And i don't know how to periodically decrease the learning rate, is it adam or other methods? – ohdoughnut May 01 '18 at 13:18
-
@ohdoughnut Are you using tensorflow? There are some built in methods for it. e.g. https://stackoverflow.com/questions/33919948/how-to-set-adaptive-learning-rate-for-gradientdescentoptimizer Although I prefer piecewise drops – geometrikal May 01 '18 at 13:29
-
-
You can decrease the learning rate continuously, or in steps (piecewise). Piecewise is a maths term that means a signal is made up of discontinuous pieces. Eg learning rate that is constant for while then drops down to a new constant. I think this is term used in the tensorflow docs, either that or periodic. Kerala should have it as it very common when using SGD – geometrikal May 01 '18 at 13:57
-
Thank you very much! I find that the keras have a function named keras.callbacks.LearningRateScheduler(schedule), it can set learning rate according to the epoch that is piecewise. I will try it. – ohdoughnut May 02 '18 at 01:53