15

I've successfully built a simple C++ app running TF Lite model by adding my sources to tensorflow/lite/examples, similarly to what the official C++ TF guide suggests for full TF. Now I want to build it as a separate project (shared library) linking to TF Lite statically and using CMake as a build system.

I tried to add a custom target to my CMakeLists.txt, which would build TF Lite with Bazel:

set(TENSORFLOW_DIR ${CMAKE_SOURCE_DIR}/thirdparty/tensorflow)
add_custom_target(TFLite
    COMMAND bazel build //tensorflow/lite:framework
    COMMAND bazel build //tensorflow/lite/kernels:builtin_ops
    WORKING_DIRECTORY ${TENSORFLOW_DIR})

I chose those Bazel targets because the BUILD file from tensorflow/lite/examples/minimal has them as dependencies, and they work for me when I build my code with Bazel within TF repo. Not sure if that's enough.

Then I manually collect include dirs (with that ugly temporarily hardcoded path) and libs:

set(TFLite_INCLUDES
    ${TENSORFLOW_DIR}
    ~/.cache/bazel/_bazel_azymohliad/ec8567b83922796adb8477fcbb00a36a/external/flatbuffers/include)

set(TFLite_LIBS
    ${TENSORFLOW_DIR}/bazel-bin/tensorflow/lite/libframework.pic.a)
    
target_include_directories(MyLib ... PRIVATE ... ${TFLite_INCLUDES})
target_link_libraries(MyLib ... ${TFLite_LIBS})

And with this configuration, I get many undefined references to TFLite stuff during linkage. I checked with nm and those symbols are indeed missing in libframework.pic.a, I found some of them in various .o files in Bazel output. Manually picking all those .o files seems wrong.

So, is it possible to link nicely to TF Lite from CMake like I'm trying to? Maybe is there some magical bazel query include_dirs(//tensorflow/lite:framework) command which would give me paths to all the necessary include dirs, and a similar command for libraries to link against so that I could pass this info to CMake?

Community
  • 1
  • 1
Andrii Zymohliad
  • 1,237
  • 1
  • 11
  • 17
  • 1
    Just a comment, I think you should be able to collect the flatbuffers headers from `bazel-genfiles/external` in the TensorFlow tree (after build) instead of `~/.cache/...`. – jdehesa Mar 12 '19 at 16:35
  • @jdehesa, thanks, that would be really useful, but for some reason, `bazel-genfiles` contains only `tensorflow` directory in my case and no `external` – Andrii Zymohliad Mar 12 '19 at 16:57
  • 1
    Ah wait maybe it's another one, have you checked `bazel-bin/external` and `bazel-tensorflow/external`? – jdehesa Mar 12 '19 at 17:07
  • 1
    Anyway about your problem, the thing is each `.a` only contains the `.c` code of its goal, not the dependencies. Also, I haven't found any good way of tracking headers either. My way around this has been adding a code file and a new goal to the TF tree (in a new subdir) with dependency to what I needed. With a `cc_library` I'm not sure if you can get a `.a` containing everything through options, with a `cc_binary` you get at least a `.params` file under `bazel-bin` with all the `.a` dependencies that you can scan, or you can your make a `.so` (personally I'm on Windows and make a DLL nowadays) – jdehesa Mar 12 '19 at 17:17
  • 1
    About the headers, I used to painfully list all the necessary paths that I needed to copy, pretty much by trial and error. What I do now is I have my own wrapper for the library (a very simple interface where I can load a `.pb` model file and run it for given inputs), hiding every native TF type (pimpl mostly). It took a number of tricks but now I only need a couple of headers of mine and this DLL. – jdehesa Mar 12 '19 at 17:22
  • 1
    This is all for regular TF btw, not TF Lite. Now there is [TensorFlow for C](https://www.tensorflow.org/install/lang_c), so that work has become kind of redundant, but I don't think there is an equivalent for Lite, so I thought the same approaches should work. – jdehesa Mar 12 '19 at 17:25
  • 1
    @jdehesa, thanks a lot for all your suggestions. Indeed, I found external headers in `bazel-tensorflow/external`. For the linkage, I really don't want to add any code to TF tree but to treat TF as a usual dependency, so I ended up adding all needed object files to `TFLite_LIBS` variable above and it works now. I made a script which goes through every undefined reference and look for object file defining it in `bazel-bin`, and after few iterations I collected everything I need. – Andrii Zymohliad Mar 13 '19 at 13:07

1 Answers1

17

I ended up listing all necessary TFLite object files manually for CMake's target_link_libraries (in the TFLite_LIBS) and it works.

I used a simple shell script to get the list of necessary object files. First I collected all undefined references from build log into a bash-array as following:

SYMBOLS=(\
    'tflite::CombineHashes('\
    'tflite::IsFlexOp('\
    'tflite::ConvertArrayToTfLiteIntArray('\
    'tflite::EqualArrayAndTfLiteIntArray('\
    ...
    'tflite::ConvertVectorToTfLiteIntArray(')

Then for every symbol in that array, I went through every *.o file in bazel build output:

for SYMBOL in $SYMBOLS[@]; do
    for OBJ in $(find -L /path/to/tensorflow/bazel-bin/ -name '*.o'); do
        nm -C $OBJ | grep "T $SYMBOL" > /dev/null && echo $OBJ
    done
done | sort | uniq

and added the output to TFLite_LIBS in CMake (with correct path prefix, of course). After that, I got a new portion of undefined references, but after a few iterations, it resolved everything.

Probably I could also obtain the full list of dependencies from *-params file from my initial in-tree build, but a quick check showed that it had some redundant items, and the script collected only the necessary ones.

For include locations, I replaced that hardcoded path to flatbuffers in bazel cache with ${TENSORFLOW_DIR}/bazel-tensorflow/external/flatbuffers/include/. Thanks jdehesa for the hint.

UPDATE:
The native build of all-inclusive TF Lite static library can be done very similar to official build instructions for RPi, iOS or ARM64 using plain old make:
1. ./tensorflow/lite/tools/make/download_dependencies.sh
2. make -f tensorflow/lite/tools/make/Makefile

The output library will be stored as <tensorflow-root>/tensorflow/lite/tools/make/gen/<platform>/lib/libtensorflow-lite.a. And the external dependencies with their headers would go into <tensorflow-root>/tensorflow/tensorflow/lite/tools/make/downloads (for example flatbuffers headers are in <tensorflow-root>/tensorflow/tensorflow/lite/tools/make/downloads/flatbuffers/include).

Guide doesn't mention that make could be called directly. There are wrapper-scripts for different cross-compilation targets, which just set appropriate variables and run make. But by default make would just do native build. This make invokation can be added as a custom command in CMakeLists.txt.

Andrii Zymohliad
  • 1,237
  • 1
  • 11
  • 17
  • When I try to use the static library that is built with the updated approach I get an undefined reference error: ./tensorflow/tensorflow/lite/tools/make/gen/linux_x86_64/lib/libtensorflow-lite.a(nnapi_implementation.o): In function `(anonymous namespace)::ASharedMemory_create(char const*, unsigned long)': nnapi_implementation.cc:(.text+0x14): undefined reference to `shm_open' Do you maybe know where this comes from? – Marcel_marcel1991 Sep 23 '19 at 12:12
  • Okay I found that there is a build script called build_lib.sh which seems to turn off nnapi. Whith this I do not get this error anymore. However I am wondering what the nnapi is useful for and if it is only supported on some platforms? – Marcel_marcel1991 Sep 23 '19 at 12:39
  • @Marcel_marcel1991, I don't really know, sorry. And I still use the first approach: build with bazel, then manually list all files for linkage. I found it easier to maintain for multiple platforms. – Andrii Zymohliad Sep 24 '19 at 17:53
  • Related to Make build: the `tensorflow/lite/tools/make/` contains only README file, stating `Using Makefile to build TensorFlow Lite is deprecated at the Aug 2021. Please use CMake or Bazel instead.` – Danijel Mar 17 '22 at 13:32
  • Last tag which contains the Make files seems to be v2.6.3: https://github.com/devialet/tensorflow/tree/v2.6.3/tensorflow/lite/tools/make – Danijel Mar 17 '22 at 13:46