I have some (old) Tensorflow model that have been created by another person.
For each of these model, I want to create frozen graph.
My models can be either Saved Model :
+--- name.txt
+--- saved_model.pb
+--- variables
| +--- variables.data-00000-of-00001
| +--- variables.index
Or Meta Graphs :
+--- export.data-00000-of-00001
+--- export.index
+--- export.meta
So, I tried to create frozen graphs using these functions:
import tensorflow as tf
import os
def frozenGraph_SavedGraph(modelFolder, outputFolder, toolsFilepath)
networkGraph = "saved_model.pb"
networkCheckpoint = "variables/variables.data-00000-of-00001"
args = "--checkpoint_version=1\
--input_graph=" + modelFolder + "/" + networkGraph + "\
--input_checkpoint=" + modelFolder + "/" + networkCheckpoint + "\
--output_graph=" + outputFolder + "\
--input_binary=true"
os.system(toolsFilepath + "/freeze_graph.py " + args)
def frozenGraph_Metagraph(modelFolder, outputFolder, toolsFilepath)
networkMeta = "export.meta"
networkCheckpoint = "export.data-00000-of-00001"
args = "--checkpoint_version=1\
--input_meta_graph=" + modelFolder + "/" + networkMeta + "\
--input_checkpoint=" + modelFolder + "/" + networkCheckpoint + "\
--output_graph=" + outputFolder + "\
--input_binary=true"
os.system(toolsFilepath + "/freeze_graph.py " + args)
toolsFilepath = os.path.dirname(tf.__file__) + "/python/tools"
savedGraphFilepath = [PATH/TO/SAVED_GRAPH]
metaGraphFilepath = [PATH/TO/META_GRAPH]
outputFolder = "/output"
frozenGraph_SavedGraph(savedGraphFilepath, outputFolder, toolsFilepath)
frozenGraph_Metagraph(metaGraphFilepath, outputFolder, toolsFilepath)
But for both of these function, I get the error message :
You need to supply the name of a node to --output_node_names.
The problem is that I don't think that I have the node names in the file that are available to me (those listed above).
Is there a way to generate frozen graphs without these informations ? (Or a way to retrieve them ?)
EDIT :
I tried to retrieve node names from the graph using this code :
def printNames_Metagraph(modelFolder):
networkMeta = "export.meta"
networkCheckpoint = "export.data-00000-of-00001"
saver = tf.train.import_meta_graph(modelFolder + "/" + networkMeta)
sess = tf.Session()
saver.restore(sess, modelFolder + "/" + networkCheckpoint)
graph = sess.graph
print([node.name for node in graph.as_graph_def().node])
But it throws a lot of error and warnings :
2019-03-14 16:40:42.973341: W tensorflow/core/framework/op_def_util.cc:357] Op TensorArray is deprecated. It will cease to work in GraphDef version 16. Use TensorArrayV3.
2019-03-14 16:40:42.973738: W tensorflow/core/framework/op_def_util.cc:357] Op TensorArrayScatter is deprecated. It will cease to work in GraphDef version 19. Use TensorArrayGradV3.
2019-03-14 16:40:42.974148: W tensorflow/core/framework/op_def_util.cc:357] Op TensorArrayRead is deprecated. It will cease to work in GraphDef version 16. Use TensorArrayReadV3.
2019-03-14 16:40:42.974740: W tensorflow/core/framework/op_def_util.cc:357] Op TensorArrayWrite is deprecated. It will cease to work in GraphDef version 16. Use TensorArrayWriteV3.
2019-03-14 16:40:42.975107: W tensorflow/core/framework/op_def_util.cc:357] Op TensorArraySize is deprecated. It will cease to work in GraphDef version 16. Use TensorArraySizeV3.
2019-03-14 16:40:42.975378: W tensorflow/core/framework/op_def_util.cc:357] Op TensorArrayGather is deprecated. It will cease to work in GraphDef version 16. Use TensorArrayGatherV3.
WARNING: Logging before flag parsing goes to stderr.
W0314 16:40:43.424403 10080 meta_graph.py:897] The saved meta_graph is possibly from an older release:
'model_variables' collection should be of type 'byte_list', but instead is of type 'node_list'.
2019-03-14 16:40:43.439121: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
W0314 16:40:43.442324 10080 deprecation.py:323] From [MY/PATH/]lib\site-packages\tensorflow\python\training\saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
2019-03-14 16:40:44.049683: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open [MY/PATH2/]/models\PATH/TO/METAGRAPH/export.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
2019-03-14 16:40:44.052551: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open [MY/PATH2/]/models\PATH/TO/METAGRAPH/export.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
2019-03-14 16:40:44.052574: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open [MY/PATH2/]/models\PATH/TO/METAGRAPH/export.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
2019-03-14 16:40:44.053005: W tensorflow/core/framework/op_kernel.cc:1431] OP_REQUIRES failed at save_restore_tensor.cc:175 : Data loss: Unable to open table file [MY/PATH2/]/models\PATH/TO/METAGRAPH/export.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
...
Traceback (most recent call last):
File "[MY/PATH/]lib\site-packages\tensorflow\python\client\session.py", line 1335, in _do_call
return fn(*args)
File "[MY/PATH/]lib\site-packages\tensorflow\python\client\session.py", line 1320, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "[MY/PATH/]lib\site-packages\tensorflow\python\client\session.py", line 1408, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.DataLossError: Unable to open table file [MY/PATH2/]/models\PATH/TO/METAGRAPH/export.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
[[{{node save/RestoreV2_311}}]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "[MY/PATH2/]Main.py", line 80, in <module>
printNames()
File "[MY/PATH2/]Main.py", line 64, in printNames
saver.restore(sess, modelFolder + "/" + networkWeightsFile)
File "[MY/PATH/]lib\site-packages\tensorflow\python\training\saver.py", line 1286, in restore
{self.saver_def.filename_tensor_name: save_path})
File "[MY/PATH/]lib\site-packages\tensorflow\python\client\session.py", line 930, in run
run_metadata_ptr)
File "[MY/PATH/]lib\site-packages\tensorflow\python\client\session.py", line 1153, in _run
feed_dict_tensor, options, run_metadata)
File "[MY/PATH/]lib\site-packages\tensorflow\python\client\session.py", line 1329, in _do_run
run_metadata)
File "[MY/PATH/]lib\site-packages\tensorflow\python\client\session.py", line 1349, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.DataLossError: Unable to open table file [MY/PATH2/]/models\PATH/TO/METAGRAPH/export.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
[[node save/RestoreV2_311 (defined at [MY/PATH2/]Main.py:62) ]]
Original stack trace for 'save/RestoreV2_311':
File "[MY/PATH2/]Main.py", line 80, in <module>
printNames()
File "[MY/PATH2/]Main.py", line 62, in printNames
saver = tf.train.import_meta_graph(modelFolder + "/" + networkArcFile)
File "[MY/PATH/]\lib\site-packages\tensorflow\python\training\saver.py", line 1445, in import_meta_graph
meta_graph_or_file, clear_devices, import_scope, **kwargs)[0]
File "[MY/PATH/]\lib\site-packages\tensorflow\python\training\saver.py", line 1467, in _import_meta_graph_with_return_elements
**kwargs))
File "[MY/PATH/]\lib\site-packages\tensorflow\python\framework\meta_graph.py", line 855, in import_scoped_meta_graph_with_return_elements
return_elements=return_elements)
File "[MY/PATH/]\lib\site-packages\tensorflow\python\util\deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "[MY/PATH/]\lib\site-packages\tensorflow\python\framework\importer.py", line 443, in import_graph_def
_ProcessNewOps(graph)
File "[MY/PATH/]\lib\site-packages\tensorflow\python\framework\importer.py", line 236, in _ProcessNewOps
for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access
File "[MY/PATH/]\lib\site-packages\tensorflow\python\framework\ops.py", line 3612, in _add_new_tf_operations
for c_op in c_api_util.new_tf_operations(self)
File "[MY/PATH/]\lib\site-packages\tensorflow\python\framework\ops.py", line 3612, in <listcomp>
for c_op in c_api_util.new_tf_operations(self)
File "[MY/PATH/]\lib\site-packages\tensorflow\python\framework\ops.py", line 3504, in _create_op_from_tf_operation
ret = Operation(c_op, self)
File "[MY/PATH/]\lib\site-packages\tensorflow\python\framework\ops.py", line 1961, in __init__
self._traceback = tf_stack.extract_stack()
Process finished with exit code 1
Notes :
- Using Tensorflow 1.13.1 with Python 3
- The models I am trying to load might use an old version of Tensorflow with the first version of checkpoints
- The models are binary files
