I'm really eager to start using Google's new Tensorflow library in C++. The website and docs are just really unclear in terms of how to build the project's C++ API and I don't know where to start.
Can someone with more experience help by discovering and sharing a guide to using tensorflow's C++ API?
To get started, you should download the source code from Github, by following the instructions here (you'll need Bazel and a recent version of GCC).
The C++ API (and the backend of the system) is in tensorflow/core. Right now, only the C++ Session interface, and the C API are being supported. You can use either of these to execute TensorFlow graphs that have been built using the Python API and serialized to a GraphDef protocol buffer. There is also an experimental feature for building graphs in C++, but this is currently not quite as full-featured as the Python API (e.g. no support for auto-differentiation at present). You can see an example program that builds a small graph in C++ here.
The second part of the C++ API is the API for adding a new OpKernel, which is the class containing implementations of numerical kernels for CPU and GPU. There are numerous examples of how to build these in tensorflow/core/kernels, as well as a tutorial for adding a new op in C++.
To add to @mrry's post, I put together a tutorial that explains how to load a TensorFlow graph with the C++ API. It's very minimal and should help you understand how all of the pieces fit together. Here's the meat of it:
Requirements:
Folder structure:
tensorflow/tensorflow/|project name|/tensorflow/tensorflow/|project name|/|project name|.cc (e.g. https://gist.github.com/jimfleming/4202e529042c401b17b7)tensorflow/tensorflow/|project name|/BUILDBUILD:
cc_binary(
name = "<project name>",
srcs = ["<project name>.cc"],
deps = [
"//tensorflow/core:tensorflow",
]
)
Two caveats for which there are probably workarounds:
https://medium.com/@jimfleming/loading-a-tensorflow-graph-with-the-c-api-4caaff88463f
If you are thinking into using Tensorflow c++ api on a standalone package you probably will need tensorflow_cc.so ( There is also a c api version tensorflow.so ) to build the c++ version you can use:
bazel build -c opt //tensorflow:libtensorflow_cc.so
Note1: If you want to add intrinsics support you can add this flags as: --copt=-msse4.2 --copt=-mavx
Note2: If you are thinking into using OpenCV on your project as well, there is an issue when using both libs together (tensorflow issue) and you should use --config=monolithic.
After building the library you need to add it to your project. To do that you can include this paths:
tensorflow
tensorflow/bazel-tensorflow/external/eigen_archive
tensorflow/bazel-tensorflow/external/protobuf_archive/src
tensorflow/bazel-genfiles
And link the library to your project:
tensorflow/bazel-bin/tensorflow/libtensorflow_framework.so (unused if you build with --config=monolithic)
tensorflow/bazel-bin/tensorflow/libtensorflow_cc.so
And when you are building your project you should also specify to your compiler that you are going to use c++11 standards.
Side Note: Paths relative to tensorflow version 1.5 (You may need to check if in your version anything changed).
Also this link helped me a lot into finding all this infos: link
First, after installing protobuf and eigen, you'd like to build Tensorflow:
./configure
bazel build //tensorflow:libtensorflow_cc.so
Then Copy the following include headers and dynamic shared library to /usr/local/lib and /usr/local/include:
mkdir /usr/local/include/tf
cp -r bazel-genfiles/ /usr/local/include/tf/
cp -r tensorflow /usr/local/include/tf/
cp -r third_party /usr/local/include/tf/
cp -r bazel-bin/libtensorflow_cc.so /usr/local/lib/
Lastly, compile using an example:
g++ -std=c++11 -o tf_example \
-I/usr/local/include/tf \
-I/usr/local/include/eigen3 \
-g -Wall -D_DEBUG -Wshadow -Wno-sign-compare -w \
-L/usr/local/lib/libtensorflow_cc \
`pkg-config --cflags --libs protobuf` -ltensorflow_cc tf_example.cpp
One alternative to using Tensorflow C++ API I found is to use cppflow.
It's a lightweight C++ wrapper around Tensorflow C API. You get very small executables and it links against the libtensorflow.so already compiled file. There are also examples of use and you use CMAKE instead of Bazel.
If you wish to avoid both building your projects with Bazel and generating a large binary, I have assembled a repository instructing the usage of the TensorFlow C++ library with CMake. You can find it here. The general ideas are as follows:
tensorflow/BUILD (the provided ones do not include all of the C++ functionality).
If you don't mind using CMake, there is also tensorflow_cc project that builds and installs TF C++ API for you, along with convenient CMake targets you can link against. The project README contains an example and Dockerfiles you can easily follow.
You can use this ShellScript to install (most) of it's dependencies, clone, build, compile and get all the necessary files into ../src/includes folder:
https://github.com/node-tensorflow/node-tensorflow/blob/master/tools/install.sh
If you don't want to build Tensorflow yourself and your operating system is Debian or Ubuntu, you can download prebuilt packages with the Tensorflow C/C++ libraries. This distribution can be used for C/C++ inference with CPU, GPU support is not included:
https://github.com/kecsap/tensorflow_cpp_packaging/releases
There are instructions written how to freeze a checkpoint in Tensorflow (TFLearn) and load this model for inference with the C/C++ API:
https://github.com/kecsap/tensorflow_cpp_packaging/blob/master/README.md
Beware: I am the developer of this Github project.
I use a hack/workaround to avoid having to build the whole TF library myself (which saves both time (it's set up in 3 minutes), disk space, installing dev dependencies, and size of the resulting binary). It's officially unsupported, but works well if you just want to quickly jump in.
Install TF through pip (pip install tensorflow or pip install tensorflow-gpu). Then find its library _pywrap_tensorflow.so (TF 0.* - 1.0) or _pywrap_tensorflow_internal.so (TF 1.1+). In my case (Ubuntu) it's located at /usr/local/lib/python2.7/dist-packages/tensorflow/python/_pywrap_tensorflow.so. Then create a symlink to this library called lib_pywrap_tensorflow.so somewhere where your build system finds it (e.g. /usr/lib/local). The prefix lib is important! You can also give it another lib*.so name - if you call it libtensorflow.so, you may get better compatibility with other programs written to work with TF.
Then create a C++ project as you are used to (CMake, Make, Bazel, whatever you like).
And then you're ready to just link against this library to have TF available for your projects (and you also have to link against python2.7 libraries)! In CMake, you e.g. just add target_link_libraries(target _pywrap_tensorflow python2.7).
The C++ header files are located around this library, e.g. in /usr/local/lib/python2.7/dist-packages/tensorflow/include/.
Once again: this way is officially unsupported and you may run in various issues. The library seems to be statically linked against e.g. protobuf, so you may run in odd link-time or run-time issues. But I am able to load a stored graph, restore the weights and run inference, which is IMO the most wanted functionality in C++.
answers above are good enough to show how to build the library, but how to collect the headers are still tricky. here I share the little script I use to copy the necessary headers.
SOURCE is the first param, which is the tensorflow source(build) direcoty;
DST is the second param, which is the include directory holds the collected headers. (eg. in cmake, include_directories(./collected_headers_here)).
#!/bin/bash
SOURCE=$1
DST=$2
echo "-- target dir is $DST"
echo "-- source dir is $SOURCE"
if [[ -e $DST ]];then
echo "clean $DST"
rm -rf $DST
mkdir $DST
fi
# 1. copy the source code c++ api needs
mkdir -p $DST/tensorflow
cp -r $SOURCE/tensorflow/core $DST/tensorflow
cp -r $SOURCE/tensorflow/cc $DST/tensorflow
cp -r $SOURCE/tensorflow/c $DST/tensorflow
# 2. copy the generated code, put them back to
# the right directories along side the source code
if [[ -e $SOURCE/bazel-genfiles/tensorflow ]];then
prefix="$SOURCE/bazel-genfiles/tensorflow"
from=$(expr $(echo -n $prefix | wc -m) + 1)
# eg. compiled protobuf files
find $SOURCE/bazel-genfiles/tensorflow -type f | while read line;do
#echo "procese file --> $line"
line_len=$(echo -n $line | wc -m)
filename=$(echo $line | rev | cut -d'/' -f1 | rev )
filename_len=$(echo -n $filename | wc -m)
to=$(expr $line_len - $filename_len)
target_dir=$(echo $line | cut -c$from-$to)
#echo "[$filename] copy $line $DST/tensorflow/$target_dir"
cp $line $DST/tensorflow/$target_dir
done
fi
# 3. copy third party files. Why?
# In the tf source code, you can see #include "third_party/...", so you need it
cp -r $SOURCE/third_party $DST
# 4. these headers are enough for me now.
# if your compiler complains missing headers, maybe you can find it in bazel-tensorflow/external
cp -RLf $SOURCE/bazel-tensorflow/external/eigen_archive/Eigen $DST
cp -RLf $SOURCE/bazel-tensorflow/external/eigen_archive/unsupported $DST
cp -RLf $SOURCE/bazel-tensorflow/external/protobuf_archive/src/google $DST
cp -RLf $SOURCE/bazel-tensorflow/external/com_google_absl/absl $DST
Tensorflow itself only provides very basic examples about C++ APIs.
Here is a good resource which includes examples of datasets, rnn, lstm, cnn and more
tensorflow c++ examples
We now provide a pre-built library and a Docker image for easy installation and usage of the TensorFlow C++ API at https://github.com/ika-rwth-aachen/libtensorflow_cc
Try it out yourself by running the example application:
git clone https://github.com/ika-rwth-aachen/libtensorflow_cc.git && \
cd libtensorflow_cc && \
docker run --rm \
--volume $(pwd)/example:/example \
--workdir /example \
rwthika/tensorflow-cc:latest \
./build-and-run.sh
While we currently only support x86_64 machines running Ubuntu, this could easily be extended to other OS and platforms in the future. Except for a some exceptions, all TensorFlow versions from 2.0.0 through 2.9.2 are available, 2.10.0 coming soon.
If you want to use the TensorFlow C++ API to load, inspect, and run saved models and frozen graphs in C++, we suggest that you also check out our helper library tensorflow_cpp.
