0

I'm still doing transfer learning using Inception V3 on tensorflow. Here is the screenshot of my training using Tensorboard.
Base training source code : https://raw.githubusercontent.com/tensorflow/hub/master/examples/image_retraining/retrain.py

My dataset is around 1 mil with 80:10:10 training:validation:test ratio.
32:128 for training batch size:validation batch size
0.001 learning rate

enter image description here

As you can see the training accuracy and validation accuracy keep fluctuating (up and down). Is there any point to continue the training? Or is there something that causes this problem that I can try to solve?

Community
  • 1
  • 1
gameon67
  • 3,981
  • 5
  • 35
  • 61

1 Answers1

0

Looks like your accuracy is moving quite a bit. What kind of optimizer are you using? Adam? In any case, it might help if you added in learning rate decay.

Leo Lee
  • 66
  • 4
  • I think this is the optimizer `optimizer = tf.train.GradientDescentOptimizer(FLAGS.learning_rate)` – gameon67 Jan 30 '19 at 09:06
  • check out this post: https://stackoverflow.com/questions/33919948/how-to-set-adaptive-learning-rate-for-gradientdescentoptimizer – Leo Lee Jan 30 '19 at 09:25
  • Hey thanks for suggesting `AdamOptimizer`. I tried it and works way better than `GradientDescent` (more stable and higher accuracy). Can you edit your answer little bit more like a real answer (explaining why adam is good), so I can accept your answer – gameon67 Feb 01 '19 at 00:04