How to save a trained tensorflow model for later use for application?

Question

I am a bit of a beginner with tensorflow so please excuse if this is a stupid question and the answer is obvious.

I have created a Tensorflow graph where starting with placeholders for X and y I have optimized some tensors which represent my model. Part of the graph is something where a vector of predictions can be calculated, e.g. for linear regression something like

y_model = tf.add(tf.mul(X,w),d)
y_vals = sess.run(y_model,feed_dict={....})

After training has been completed I have acceptable values for w and d and now I want to save my model for later. Then, in a different python session I want to restore the model so that I can again run

## Starting brand new python session
import tensorflow as tf
## somehow restor the graph and the values here: how????
## so that I can run this:
y_vals = sess.run(y_model,feed_dict={....})

for some different data and get back the y-values.

I want this to work in a way where the graph for calculating the y-values from the placeholders is also stored and restored - as long as the placeholders get fed the correct data, this should work transparently without the user (the one who applies the model) needing to know what the graph looks like).

As far as I understand tf.train.Saver().save(..) only saves the variables but I also want to save the graph. I think that tf.train.export_meta_graph could be relevant here but I do not understand how to use it correctly, the documentation is a bit cryptic to me and the examples do not even use export_meta_graph anywhere.

Answer 1

From the docs, try this:

# Create some variables.
v1 = tf.Variable(..., name="v1")
v2 = tf.Variable(..., name="v2")
...
# Add an op to initialize the variables.
init_op = tf.global_variables_initializer()

# Add ops to save and restore all the variables.
saver = tf.train.Saver()

# Later, launch the model, initialize the variables, do some work, save the
# variables to disk.
with tf.Session() as sess:
  sess.run(init_op)
  # Do some work with the model.
  ..
  # Save the variables to disk.
  save_path = saver.save(sess, "/tmp/model.ckpt")
  print("Model saved in file: %s" % save_path)

You can specify the path.

And if you want to restore the model, try:

with tf.Session() as sess:
    saver = tf.train.import_meta_graph('/tmp/model.ckpt.meta')
    saver.restore(sess, "/tmp/model.ckpt")

Answer 2

Saving Graph in Tensorflow:

import tensorflow as tf

# Create some placeholder variables
x_pl = tf.placeholder(..., name="x")
y_pl = tf.placeholder(..., name="y")

# Add some operation to the Graph
add_op = tf.add(x, y)

with tf.Session() as sess:

    # Add variable initializer
    init = tf.global_variables_initializer()

    # Add ops to save variables to checkpoints
    # Unless var_list is specified Saver will save ALL named variables
    # in Graph
    # Optionally set maximum of 3 latest models to be saved
    saver = tf.train.Saver(max_to_keep=3)

    # Run variable initializer
    sess.run(init)

    for i in range(no_steps):
        # Feed placeholders with some data and run operation
        sess.run(add_op, feed_dict={x_pl: i+1, y_pl: i+5})
        saver.save(sess, "path/to/checkpoint/model.ckpt", global_step=i)

This will save the following files:

1) Meta Graph

.meta file:

• MetaGraphDef protocol buffer representation of MetaGraph which saves the complete Tf Graph structure i.e. the GraphDef that describes the dataflow and all metadata associated with it e.g. all variables, operations, collections, etc.

• importing the graph structure will recreate the Graph and all its variables, then the corresponding values for these variables can be restored from the checkpoint file

• if you don't want to restore the Graph however you can reconstruct all of the information in the MetaGraphDef by re-executing the Python code that builds the model n.b. you must recreate the EXACT SAME variables first before restoring their values from the checkpoint

• since Meta Graph file is not always needed, you can switch off writing the file in saver.save using write_meta_graph=False

2) Checkpoint files

.data file:

• binary file containing VALUES of all saved variables outlined in tf.train.Saver() (default is all variables)

.index file:

• immutable table describing all tensors and their metadata checkpoint file:

• keeps a record of latest checkpoint files saved

Restoring Graph in Tensorflow:

import tensorflow as tf

latest_checkpoint = tf.train.latest_checkpoint("path/to/checkpoint")

# Load latest checkpoint Graph via import_meta_graph:
#   - construct protocol buffer from file content
#   - add all nodes to current graph and recreate collections
#   - return Saver
saver = tf.train.import_meta_graph(latest_checkpoint + '.meta')

# Start session
with tf.Session() as sess:

    # Restore previously trained variables from disk
    print("Restoring Model: {}".format("path/to/checkpoint"))
    saver.restore(sess, latest_checkpoint)

    # Retrieve protobuf graph definition
    graph = tf.get_default_graph()

    print("Restored Operations from MetaGraph:")
    for op in graph.get_operations():
       print(op.name)

    # Access restored placeholder variables
    x_pl = graph.get_tensor_by_name("x_pl:0")
    y_pl = graph.get_tensor_by_name("y_pl:0")

    # Access restored operation to re run
    accuracy_op = graph.get_tensor_by_name("accuracy_op:0")

This is just a quick example with the basics, for a working implementation see here.

Answer 3

In order to save the graph, you need to freeze the graph. Here is the python script for freezing the graph : https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py

Here is a code snippet for freezing graph:

from tensorflow.python.tools import freeze_graph
freeze_graph.freeze_graph(input_graph_path, input_saver_def_path,
                            input_binary, checkpoint_path,  output_node
                            restore_op_name, filename_tensor_name,
                            output_frozen_graph_name, True, "")

where output node corresponds to output tensor variable.

output = tf.nn.softmax(outer_layer_name,name="output")