4

Even though there are plenty of questions here on SO raising the issue of reusing trained tensorflow models it is still a challenge to use one of the most mainstream model Inception-v3 fine-tuned on a custom dataset to just predict probabilities for some single image.

After having done some research on this topic (the most similar SO-thread is sured to be Tensorflow: restoring a graph and model then running evaluation on a single image) I can conclude that having frozen graph.pb file of some trained model is like having a holy grail because you don't need to rebuild graph, choose tensors to restore or whatever — you just call tf.import_graph_def and take the output layer you need via sess.graph.get_tensor_by_name.

But the problem is that in the examples provided with tensorflow (e.g. classify_image.py), such «frozen graph»'s have nicely prepared input and output points like DecodeJpeg/contents:0 and softmax:0 respectively where you can feed your custom images and retrieve the answers from, while you don't have such nice entry points when working with custom fine-tuned model.

For example, fine-tuned Inception-v3 model frozen graph will have FIFOQueue, QueueDequeueMany and similar dozen of tensors before actual convolutional layers to read batches from TFRecord's and the output tensor will look like tower_0/logits/predictions with unusable shape containing batch size, so you just don't have appropriate points to feed a new jpeg image in and get predictions out.

Is there any success story covering usage of such batches-fed fine-tuned models with new images? Or maybe some ideas on changing input pack of TFRecord/batch nodes to JPEG one?

P.S. There is also an alternative for running pretrained models such as TF Serving, but building a huge github repo with plenty of dependencies for every other step seems overwhelming to me.

Community
  • 1
  • 1
Evgeny
  • 51
  • 6
  • You need to load a purposefully-built inference graph instead of the training graph. Slim, tf.contrib.learn.Estimator, keras, and other frameworks all allow you to build a nice inference graph. – Alexandre Passos Mar 08 '17 at 21:52

1 Answers1

0

I was not using the Inception-v3 model, but I have found a work around for a similar situation. For training, I use a custom multiprocess/multithread setup to load my samples into batches and feed them into a FIFOQueue. Running inference on the frozen graph always hung indefinitely. Here is my approach:

Creating frozen inference model:

  1. Build a completely separate inference graph. Create placeholders [in1,in2,...] for your inputs (with shape corresponding to 1 sample) and create your model the same way you do for training. The output(s) of your model will be referred to as [out1,out2...] hereafter.

  2. Use tf.train.Saver() to load your trained model parameters (for this to work, the names in your new model must match those of your training model). Something like:

    loader = tf.train.Saver()
graph = tf.get_default_graph()
input_graph_def = graph.as_graph_def()
with tf.Session() as sess:
    loader.restore(sess, input_checkpoint)

  3. Create a frozen graph:

    frozen_graph_def = graph_util.convert_variables_to_constants(
    sess,
    input_graph_def,
    output_node_names)

  4. Optimize model:

    optimized_graph_def = optimize_for_inference_lib.optimize_for_inference(
    frozen_graph_def,
    input_node_names,
    output_node_names, tf.float32.as_datatype_enum)

  5. Save model:

    with tf.gfile.GFile(filename, "wb") as f:
    f.write(optimized_graph_def.SerializeToString())

Performing inference with frozen model:

  1. Load model into graph

    with tf.gfile.GFile(frozen_graph_filename, "rb") as f:
    graph_def = tf.GraphDef()
    graph_def.ParseFromString(f.read())
with tf.Graph().as_default() as graph:
    tf.import_graph_def(
        graph_def,
        input_map=None,
        return_elements=None,
        name='',
        op_dict=None,
        producer_op_list=None
    )

  2. Access your inputs/outputs:

    in1 = graph.get_tensor_by_name(input_tensor_names[0])
in2 = graph.get_tensor_by_name(input_tensor_names[1])
...
out1 = graph.get_tensor_by_name(output_tensor_names[0])
...

  3. Run inference:

    with tf.Session(graph=graph) as sess:
    sess.run([out1], feed_dict={in1: {data}, in2: {data})

Tip: How to get the input/output node/tensor names:

inputs = [in1, in2...]
outputs = [out1, out2...]

output_tensor_names = [n.name for n in outputs]
input_tensor_names = [n.name for n in inputs]

output_node_names = [n[:str(n).find(":")] for n in output_tensor_names]
input_node_names = [n[:str(n).find(":")] for n in input_tensor_names]

I hope this helps.

user2906021
  • 11
  • 1