Restoring model take to long

Question

I have a problem restring my model. I trained model and saved a model using this code. I'm not really sure if this is the proper method I would be grateful for suggestions. The problem occurs when I'm trying to restore model. I need it only to predict, it won't be furder trained. It takes forever to restore parameters from the model. How can I improve my model saver or model restorer to make it quick, under the assumption I need it only for predicting.

X = tf.placeholder(tf.float32, [None, 56, 56, 1])
Y_ = tf.placeholder(tf.float32, [None, 36])

L1 = 432
L2 = 72
L3 = 36

W1 = tf.Variable(tf.truncated_normal([3136, L1], stddev=0.1))
b1 = tf.Variable(tf.zeros([L1]))
W2 = tf.Variable(tf.truncated_normal([L1, L2], stddev=0.1))
b2 = tf.Variable(tf.zeros([L2]))
W3 = tf.Variable(tf.truncated_normal([L2, L3], stddev=0.1))
b3 = tf.Variable(tf.zeros([L3]))

XX = tf.reshape(X, [-1, 3136])

Y1 = tf.nn.sigmoid(tf.matmul(XX, W1) + b1)
Y1 = tf.nn.dropout(Y1, keep_prob=0.8)
Y2 = tf.nn.sigmoid(tf.matmul(Y1, W2) + b2)
Y2 = tf.nn.dropout(Y2, keep_prob=0.8)
Ylogits = tf.matmul(Y2, W3) + b3
Y = tf.nn.softmax(Ylogits)

cross_entropy = tf.nn.softmax_cross_entropy_with_logits_v2(logits=Ylogits, labels=Y_)
cross_entropy = tf.reduce_mean(cross_entropy) * 100
correct_prediction = tf.equal(tf.argmax(Y, 1), tf.argmax(Y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
train_step = tf.train.GradientDescentOptimizer(0.0001).minimize(cross_entropy)

allweights = tf.concat([tf.reshape(W1, [-1]), tf.reshape(W2, [-1]), tf.reshape(W3, [-1])], 0)
allbiases = tf.concat([tf.reshape(b1, [-1]), tf.reshape(b2, [-1]), tf.reshape(b3, [-1])], 0)

init = tf.global_variables_initializer()

saver = tf.train.Saver()

def next_batch(x, y, batch, step):
    x_temp = x[cur_step:(step+batch)]
    y_temp = np.squeeze(y[step:(step + batch)])
    return x_temp, y_temp


with tf.Session() as sess:
    sess.run(init)
    cur_step = 0
    for i in range(NUM_ITERS + 1):
        batch_X, batch_Y = next_batch(train_xx, train_yy, BATCH, cur_step)
        if i % DISPLAY_STEP == 0:
            acc_trn, loss_trn, w, b = sess.run([accuracy, cross_entropy, allweights, allbiases], feed_dict={X: batch_X, Y_: batch_Y})
            acc_tst, loss_tst = sess.run([accuracy, cross_entropy], feed_dict={X: test_xx, Y_: test_yy})

        sess.run(train_step, feed_dict={X: batch_X, Y_: batch_Y})
    save_path = saver.save(sess, "abc/model")

Restore:

X = tf.placeholder(tf.float32, [None, 56, 56, 1])
Y_ = tf.placeholder(tf.float32, [None, 36])

L1 = 432
L2 = 72
L3 = 36

W1 = tf.Variable(tf.truncated_normal([3136, L1], stddev=0.1))
b1 = tf.Variable(tf.zeros([L1]))

W2 = tf.Variable(tf.truncated_normal([L1, L2], stddev=0.1))
b2 = tf.Variable(tf.zeros([L2]))

W3 = tf.Variable(tf.truncated_normal([L2, L3], stddev=0.1))
b3 = tf.Variable(tf.zeros([L3]))


XX = tf.reshape(X, [-1, 3136])

Y1 = tf.nn.sigmoid(tf.matmul(XX, W1) + b1)
Y1 = tf.nn.dropout(Y1, keep_prob=0.8)
Y2 = tf.nn.sigmoid(tf.matmul(Y1, W2) + b2)
Y2 = tf.nn.dropout(Y2, keep_prob=0.8)
Ylogits = tf.matmul(Y2, W3) + b3
Y = tf.nn.softmax(Ylogits)

with tf.Session() as sess:
    saver = tf.train.Saver()
    saver = tf.train.import_meta_graph('model.meta')
    saver.restore(sess, 'model')

EDIT: Maybe a fact that model is trained using Google Colab's GPU and I'm restoring it on my PC is important.

score 1 · Accepted Answer · answered May 23 '18 at 12:14

Its a duplicate of: Tensorflow: how to save/restore a model?.

Your saving of the model is right but not your restoring. What you are doing is trying to create a new graph with the same nodes as the saved model, instead of restoring it from the saved graph. The following steps should address your issue on how to restore a model:

#Start with resetting the default graph
tf.reset_default_graph()

with tf.Session() as sess:

   # Nodes:Before loading the graph
   print([n.name for n in tf.get_default_graph().as_graph_def().node])
   # Output is [] as no graph is loaded yet.

   # First let's load meta graph 
   saver = tf.train.import_meta_graph("abc/model.meta")
   # Nodes:after loading the graph'
   print([n.name for n in tf.get_default_graph().as_graph_def().node])
   # Output is [save/RestoreV2/shape_and_slices', 'save/RestoreV2/tensor_ ...]

   # The above step doesnt load the weights, can be checked by
   print(sess.run('Variable_1:0'))
   # Error: attempting to use uninitialized graph.

   #load the weights 
   saver.restore(sess,tf.train.latest_checkpoint('./abc/'))
   print(sess.run('Variable_1:0'))
   # Output: [-2.80421402e-04  3.53254407e-04 ...]

Now we have the nodes loaded and ready, you need to access some of them for inference. But since the nodes are not named properly, its not easy to figure out which node are inputs and outputs. To avoid this you need to name the tensors/ops when saving the model properly using the name argument like:

X = tf.placeholder(tf.float32, [None, 56, 56, 1], name='X')
Y = tf.identity(f.nn.softmax(Ylogits), name='logits').

In your inference graph once you have loaded the graph and weights, you can get these tensors using get_tensor_by_name:

with tf.Session() as sess:

   #Load the graph and weights as above
   ....

   graph = tf.get_default_graph()
   X_infer = graph.get_tensor_by_name('X:0')
   Y_infer = graph.get_tensor_by_name('logits:0')
   sess.run(Y_infer,{X_infer:new_input}

Restoring model take to long

1 Answers1

Linked