I've followed along with the linked posts and finally managed to load my trained tensorflow model's graph+weights into C++ without it throwing errors at me, but it seems to not properly load the weights. It's possible I'm missing a step, most likely in the inference section.
I've included a fully functional minimal working example below. In order to run it you'll (probably) need Python + 1.4.0 < Tensorflow <= 1.7.0, Tensorflow for C++, and keras. Any insights into the issue I'm having would be very happily received, I've been in an uphill battle with tf for 2+weeks now.
Python code which will "train" a model and save it + weights as shown here:
from __future__ import absolute_import, division, print_function
import os
import tensorflow as tf
import numpy as np
import random
import keras
from keras import backend as K
# Set it manually so C++ interface can use mem growth
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
sess = tf.Session(config=config)
K.set_session(sess)
model = tf.keras.Sequential([
tf.keras.layers.Convolution2D(64, (5, 5), input_shape=(7, 19, 19), data_format='channels_first', name='Input'),
tf.keras.layers.ZeroPadding2D(padding =(4, 4), data_format='channels_first', name='Pad0'),
tf.keras.layers.BatchNormalization(axis=1, momentum=0.99, name='Norm0'),
tf.keras.layers.Dropout(0.25, name='Drop0'),
tf.keras.layers.Flatten(name='Flatten0'),
tf.keras.layers.Dense(361, activation='softmax', name='Output'),
])
X = np.ones((25, 7, 19, 19))
Y = np.zeros((25, 361))
optimizer = tf.train.AdamOptimizer(learning_rate=0.0018)
model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
x = np.zeros((1, 7, 19, 19))
print(model.predict(x))
model.fit(X, Y, 1, 5, 2)
print(model.predict(x)) # Just to verify output is different after training
K.set_learning_phase(0)
sess.run(tf.global_variables_initializer())
saver = tf.train.Saver(tf.global_variables())
saver.save(sess, save_path='./models/myModel')
// Just used to look at graph structure
tf.train.write_graph(sess.graph, '.', './models/graph.pb', as_text=True)
And C++ code, currently using tf 1.5.0:
Code to load the model as shown here
#define _SILENCE_ALL_CXX17_DEPRECATION_WARNINGS
#define COMPILER_MSVC
#define NOMINMAX
#include <iomanip>
#include <tensorflow\core\public\session.h>
#include <tensorflow\core\protobuf\meta_graph.pb.h>
#include <tensorflow\core\framework\tensor.h>
#include <tensorflow\cc\ops\standard_ops.h>
namespace tf = tensorflow;
tf::Status status;
tf::Session* session;
tf::SessionOptions options;
tf::MetaGraphDef graphDef;
std::string pathToGraph = "models/myModel.meta";
std::string pathToCheckpoint = "models/myModel";
int main()
{
options.config.mutable_gpu_options()->set_allow_growth(true);
options.config.mutable_gpu_options()->set_per_process_gpu_memory_fraction(1.0);
session = tf::NewSession(options);
if (!session)
throw std::runtime_error("Could no create Tensorflow Session!");
// Read in the protobuf
status = tf::ReadBinaryProto(tf::Env::Default(), pathToGraph, &graphDef);
if (!status.ok())
throw std::runtime_error("Error reading graph: " + status.ToString());
status = session->Create(graphDef.graph_def());
if (!status.ok())
throw std::runtime_error("Error creating graph: " + status.ToString());
// Read the weights
tf::Tensor checkpointPathTensor(tf::DT_STRING, tf::TensorShape());
checkpointPathTensor.scalar<std::string>()() = pathToCheckpoint;
const auto fileTensorName = graphDef.saver_def().filename_tensor_name();
const auto restoreOpName = graphDef.saver_def().restore_op_name();
status = session->Run(
{ { fileTensorName, checkpointPathTensor }, },
{},
{ restoreOpName },
nullptr
);
if (!status.ok())
throw std::runtime_error("Error loading checkpoint from " + pathToCheckpoint + ": " + status.ToString());
...
Followed by the code to run inference with the loaded/trained model
float inData[2527] = { 0.f };
static const std::string inputName = "Input_input";
static const std::string outputName = "Output/Softmax";
static const auto shape = tf::TensorShape({ 1, 7, 19, 19});
tf::Tensor input(tf::DT_FLOAT, shape);
std::copy_n(inData, 2527, input.flat<float>().data());
std::vector<tf::Tensor> outputs;
status = session->Run({ { inputName, input } }, { outputName }, {}, &outputs);
tf::TTypes<float>::Flat flatOut = outputs[0].flat<float>();
for (int i = 0; i < 361; ++i)
{
if (i % 19 == 0)
std::cout << '\n';
std::cout << std::setw(8) << std::fixed << std::setprecision(8) << flatOut(i) << ", ";
}
std::cout << '\n';
}
For me, when I run this everything works as it should in the Python section but silently fails to load a trained graph (or does something that looks similar) in the C++ section. When running the C++ section the model outputs exactly the same output as the non-trained Python model does. Is there some step I'm missing here?
Sample Python output before model training:
[[0.00277008 0.00277008 0.00277008 0.00277008 0.00277008 0.00277008
0.00277008 0.00277008 0.00277008 0.00277008 0.00277008 ...etc ]]
Sample output after training in Python:
.00387822 0.00228055 0.0018196 0.0014322 0.00266262 0.00234695
0.0026182 0.00322318 0.00252047 0.00353322 0.00342526 ...etc ]]
Sample output after loading model and running in C++ with identical inputs:
0.00277008, 0.00277008, 0.00277008, 0.00277008, ... etc
Ideally I'd like to be seeing C++ output that is identical to the Python output after training!
