Using tensorflow in C++ on Windows

Question

I know there are ways of using Tensorflow in C++ they even have a documentation for it but I can seem to be able to get the library for it. I've checked the build from source instructions but it seems to builds a pip package rather than a library I can link to my project. I also found a tutorial but when I tried it out I ran out of memory and my computer crashed. My question is, how can I actually get the C++ library to work on my project? I do have these requirements, I have to work on windows with Visual Studio in C++. What I would love to is if I could get a pre-compiled DLL that I could just link but I haven't found such a thing and I'm open to other alternatives.

Answer 1

I can't comment so I am writing this as an answer. If you don't mind using Keras, you could use the package frugally deep. I haven't seen a library myself either, but I came across frugally deep and it seemed easy to implement. I am currently trying to use it, so I cannot guarantee it will work.

Answer 2

You could check out neural2D from here:

https://github.com/davidrmiller/neural2d

It is a neural network implementation without any dependent libraries (all written from scratch).

Answer 3

I would say that the best option is to use cppflow, an easy wrapper that I created to use Tensorflow from C++ easily.

You won't need to install anything, just download the TF C API, and place it somewhere in your computer. You can take a look to the docs to see how to do it, and how to use the library.

Answer 4

The answer seems to be that it is hard :-(

Try this to start. You can follow the latest instructions for building from source on Windows up to the point of building the pip package. But don't do that - do this/these instead:

bazel -config=opt //tensorflow:tensorflow.dll 
bazel -config=opt //tensorflow:tensorflow.lib
bazel -config=opt tensorflow:install_headers

That much seems to work fine. The problems really start when you try to use Any of the header files - you will probably get compilation errors, at least with TF version >= 2.0. I have tried:

• Build the label_image example (instructions in the readme.md file)
  • It builds and runs fine on Windows, meaning all the headers and source are there somewhere
  • Try incorporating that source into Windows console executable: runs into compiler errors due to conflicts with std::min & std::max, probably due to Windows SDK.
• Include c_api.h in a Windows console application: won't compile.
• Include TF-Lite header files: won't compile.

There is little point investing the lengthy compile time in the first two bazel commands if you can't get the headers to compile :-(

You may have time to invest in resolving these errors; I don't. At this stage Tensorflow lacks sufficient support for Windows C++ to rely on it, particularly in a commercial setting. I suggest exploring these options instead:

• If TF-Lite is an option, watch this
• Windows ML/Direct ML (requires conversion of TF models to ONNX format)
• CPPFlow
• Frugally Deep
• Keras2CPP

UPDATE: having explored the list above, I eventually found the following worked best in my context (real-time continuous item recognition):

• convert models to ONNX format (use tf2onnx or keras2onnx
• use Microsoft's ONNX runtime

Even though Microsoft recommends using DirectML where milliseconds matter, the performance of ONNX runtime using DirectML as an execution provider means we can run a 224x224 RGB image through our Intel GPU in around 20ms, which is quick enough for us. But it was still hard finding our way to this answer