1

I'm learning TensorFlow and trying to apply it on MNIST database, and surprised to see already in 2nd epoch (e1 in the screenshot) the accuracy on whole test set reaches 95% from just 9.4% during e0 (before any updates to weight & bias!)!

Is it possible such a sharp rise in accuracy?

Considering batch size of 100 & mnist training samples # = 55000. there must have been 550 updates before e1.enter image description here

Is there anything wrong i'm doing ?

Thanks a lot in advance!

Vibhor Jain
  • 1,396
  • 1
  • 11
  • 32

1 Answers1

0

There is nothing wrong with your code as you are printing the test accuracy and it is giving good accuracy soon. Also your training loss is decreasing.

If at all you are interested in reducing the pace of learning, you can reduce the learning rate (say 0.000003). You may even experiment with reducing the number of layers or number of units in hidden layers if at all you would like the network to learn slowly.

This is the accuracy I got with your code with learning rate of 0.000003 for the first 10 epochs.

 e0_0, loss: 2.3103301525115967, accuracy: 0.09469999372959137
 e1_550, loss: 2.2732467651367188, accuracy: 0.11469998955726624
 e2_1100, loss: 2.233933210372925, accuracy: 0.1590999811887741
 e3_1650, loss: 2.185560941696167, accuracy: 0.2231999933719635
 e4_2200, loss: 2.1254842281341553, accuracy: 0.31909993290901184
 e5_2750, loss: 2.051886558532715, accuracy: 0.40129992365837097
 e6_3300, loss: 1.9651522636413574, accuracy: 0.4681999087333679
 e7_3850, loss: 1.8645144701004028, accuracy: 0.5381998419761658
 e8_4400, loss: 1.750971794128418, accuracy: 0.6107000112533569
 e9_4950, loss: 1.6259007453918457, accuracy: 0.6584999561309814
 e10_5500, loss: 1.4947171211242676, accuracy: 0.6923999786376953
Prasad
  • 5,946
  • 3
  • 30
  • 36
  • Thanks a lot! I was bit stumped to see such a sharp improvement, so suddenly! too good to believe.. thought i'd ask experts opinions anything wrong I was doing ?;) – Vibhor Jain Nov 29 '17 at 03:29
  • This doesn't answer the question. Plus, "if it works on MNIST there are no bugs and no need to worry" is a *very* harmful thought: see [this](https://github.com/zalandoresearch/fashion-mnist#to-serious-machine-learning-researchers). The deep learning scene doesn't need more encouragement for empirism, rather the total opposite! – fr_andres Apr 08 '18 at 14:51
  • I don't see that this answer stated: "if it works on MNIST there are no bugs and no need to worry". That's a fine article to share for sure, but I don't see how it relates to this answer. This answer looks plenty helpful to me. – David Parks Apr 08 '18 at 17:41
  • I must have totally misread something, sorry. I will correct the downvote – fr_andres Apr 08 '18 at 23:58
  • Still, I think my critique holds as for saying "it learns, therefore code is fine" is not the way one wants to validate any machine learning setup.The link backs up the idea that pretty much anything works on MNIST, and that this criterion is not good enough. But I agree with you that it isn't required – fr_andres Apr 09 '18 at 00:00