4

How does one use local variables in a saved Tensorflow model? In particular, I want to use tf.contrib.metrics.streaming_auc. Normally, one does sess.run(tf.local_variables_initializer()) before one can use this op (it seems to have local variables for true positives, etc.). This works fine when I've just made a model. However, when I reload a model, running sess.run(tf.local_variables_initializer()) gives me AttributeError: 'Tensor' object has no attribute 'initializer'. If I try to use the auc op without initializing local variables, I get FailedPreconditionError: Attempting to use uninitialized value auc/true_positives. Do I need to acquire/save references to the local variables used in the auc op and initialize them directly somehow when I reload the model?

Here is an example that fails:

inputs = tf.random_normal([10, 2])
labels = tf.reshape(tf.constant([1] * 5 + [0] * 5), (-1, 1))
logits = tf.layers.dense(inputs, 1)
pred = tf.sigmoid(logits)
auc_val, auc_op = tf.contrib.metrics.streaming_auc(pred, labels)

sess = tf.Session()
sess.run([tf.global_variables_initializer(), tf.local_variables_initializer()])

print(sess.run(auc_op)) # some terrible AUC, but it works
saver = tf.train.Saver()
saver.save(sess, 'test.model')

Then, in a new process:

sess = tf.Session()

saver = tf.train.import_meta_graph('test.model.meta')
saver.restore(sess, 'test.model')
auc_op = tf.get_default_graph().get_operation_by_name('auc/update_op')
print(sess.run(auc_op))
# FailedPreconditionError: Attempting to use uninitialized value auc/true_positives

Note also that if you do sess.run(tf.local_variables_initializer()) to try to initialize, then you get the other error mentioned above.

One can make a new auc op after loading, but it would be nicer not to have to. Also, for that new auc op, you have to initialize the local variables, so you have to filter those out from the un-intializable Tensors left over in local_variables from the previous auc op.

Nathan
  • 9,651
  • 4
  • 45
  • 65
  • Not entirely sure on this, but it MAY have something to do with the order in which your variables were declared in the original model. Make sure that your variables are declared BEFORE the Saver, so this way the Saver includes them in the graph. In my case i was getting the 'Attempting to use uninitialized value' exactly for this reason. More on this is in this answer: http://stackoverflow.com/questions/43663795/tf-lstm-save-state-from-training-session-for-prediction-session-later/43670332#43670332 – VS_FF May 13 '17 at 12:12
  • @VS_FF I do define all of my variables before I create the Saver. I don't initialize them until afterwards, but I don't think that should matter? – Nathan May 13 '17 at 21:14
  • I think in most cases the initialize ops are run after the Saver declaration, so shouldn't be an issue. Not sure what else to try. maybe one clunky workaround would be to assign the local variable in question to a global one before saving and try restoring that one instead? Hopefully others would have better suggestions. Also, in that discussion that I linked earlier, there is a nice example of showing all variables that have been saved in the graph; perhaps rune that to see that yours is indeed among the saved ones? – VS_FF May 13 '17 at 22:14

0 Answers0