1

I am a beginner in TensorFlow, currently training a CNN.

I am using Saver in order to save the parameters used by the model, but I am having concerns whether this would itself store all the Variables used by the model, and is sufficient to restore the values to re-run the program for performing classification/testing on the trained network.

Let us look at the famous example MNIST given by TensorFlow.

In the example, we have bunch of Convolutional blocks, all of which have weight, and bias variables that gets initialised when the program is run.

W_conv1 = init_weight([5,5,1,32])
b_conv1 = init_bias([32])

After having processed several layers, we create a session, and initialise all the variables added to the graph. 

sess = tf.Session()
sess.run(tf.initialize_all_variables())
saver = tf.train.Saver()

Here, is it possible to comment the saver.save code, and replace it by saver.restore(sess,file_path) after the training, in order to restore the weight, bias, etc., parameters back to the graph? Is this how it should be ?

for i in range(1000):
 ...

  if i%500 == 0:
    saver.save(sess,"model%d.cpkt"%(i))

I am currently training on large dataset, so terminating, and restarting the training is a waste of time, and resources so I request someone to please clarify before the I start the training.

VM_AI
  • 1,132
  • 4
  • 13
  • 25
  • It's a bit unclear what you are asking. "to comment the saver.save code, and replace it by saver.restore(sess,file_path)" You don't want to store your trained values and reset from previous training (by restoring)? "so terminating, and restarting the training is a waste of". It means you want to save the model once when you finish all training? – Sung Kim Apr 23 '16 at 23:22
  • @Sung Kim: The answer is yes to your latter question. It is not my intent to restart the training with the stored values, rather to simply save the model once for all after complete training. Because in Matlab, this is quite straightforward, and as a matter of fact, this is the first time I am programming Python, and with TensorFlow, so I do not know whether there is any other elegant way to save the parameters. – VM_AI Apr 24 '16 at 10:13

2 Answers2

4

If you want to save the final result only once, you can do this:

with tf.Session() as sess:
  for i in range(1000):
    ...


  path = saver.save(sess, "model.ckpt") # out of the loop
  print "Saved:", path

In other programs, you can load the model using the path returned from saver.save for prediction or something. You can see some examples at https://github.com/sugyan/tensorflow-mnist.

Sung Kim
  • 8,417
  • 9
  • 34
  • 42
2

Based on the explanation in here and Sung Kim solution I wrote a very simple model exactly for this problem. Basically in this way you need to create an object from the same class and restore its variables from the saver. You can find an example of this solution here.

Phoenix666
  • 183
  • 1
  • 14