TensorFlow static C API library - how to link with 10 sub-dependencies?

Question

I am trying to link with static C API version of the TensorFlow library. I built the static library using the following commands:

// get the sources
git clone https://github.com/tensorflow/tensorflow.git tensorflow_src

// create a build directory
mkdir builddir
cd builddir

// build the lib using CMake
cmake -S ../tensorflow_src/tensorflow/lite/c -DTFLITE_C_BUILD_SHARED_LIBS:BOOL=OFF
cmake --build . -j

This builds the libtensorflow-lite.a. However, the libtensorflow-lite.a is not self-contained and has its own set of 10 dependencies, stated here in the CMake file:

# TensorFlow Lite dependencies.

find_package(absl REQUIRED)
find_package(eigen REQUIRED)
find_package(farmhash REQUIRED)
find_package(fft2d REQUIRED)
find_package(flatbuffers REQUIRED)
find_package(gemmlowp REQUIRED)
find_package(neon2sse REQUIRED)
find_package(clog REQUIRED)
find_package(cpuinfo REQUIRED)  #CPUINFO is used by XNNPACK and RUY library
find_package(ruy REQUIRED)

The question is, how do I find out the .a names of the required sub-libraries?

I used find ./builddir -type f -name "*.a" to list the libraries built by CMake, and expected roughly 10 libs, but the actual list is too long:

./_deps/xnnpack-build/libXNNPACK.a
./_deps/ruy-build/ruy/libruy_pack_avx2_fma.a
./_deps/ruy-build/ruy/libruy_have_built_path_for_avx2_fma.a
./_deps/ruy-build/ruy/libruy_block_map.a
./_deps/ruy-build/ruy/libruy_system_aligned_alloc.a
./_deps/ruy-build/ruy/libruy_have_built_path_for_avx512.a
./_deps/ruy-build/ruy/profiler/libruy_profiler_instrumentation.a
./_deps/ruy-build/ruy/libruy_trmul.a
./_deps/ruy-build/ruy/libruy_cpuinfo.a
./_deps/ruy-build/ruy/libruy_blocking_counter.a
./_deps/ruy-build/ruy/libruy_pack_arm.a
./_deps/ruy-build/ruy/libruy_apply_multiplier.a
./_deps/ruy-build/ruy/libruy_kernel_avx2_fma.a
./_deps/ruy-build/ruy/libruy_prepacked_cache.a
./_deps/ruy-build/ruy/libruy_tune.a
./_deps/ruy-build/ruy/libruy_context_get_ctx.a
./_deps/ruy-build/ruy/libruy_have_built_path_for_avx.a
./_deps/ruy-build/ruy/libruy_ctx.a
./_deps/ruy-build/ruy/libruy_wait.a
./_deps/ruy-build/ruy/libruy_allocator.a
./_deps/ruy-build/ruy/libruy_context.a
./_deps/ruy-build/ruy/libruy_kernel_avx.a
./_deps/ruy-build/ruy/libruy_prepare_packed_matrices.a
./_deps/ruy-build/ruy/libruy_pack_avx512.a
./_deps/ruy-build/ruy/libruy_kernel_arm.a
./_deps/ruy-build/ruy/libruy_denormal.a
./_deps/ruy-build/ruy/libruy_kernel_avx512.a
./_deps/ruy-build/ruy/libruy_frontend.a
./_deps/ruy-build/ruy/libruy_pack_avx.a
./_deps/ruy-build/ruy/libruy_thread_pool.a
./_deps/flatbuffers-build/libflatbuffers.a
./_deps/fft2d-build/libfft2d_fftsg2d.a
./_deps/fft2d-build/libfft2d_fftsg.a
./_deps/farmhash-build/libfarmhash.a
./_deps/clog-build/libclog.a
./_deps/abseil-cpp-build/absl/synchronization/libabsl_graphcycles_internal.a
./_deps/abseil-cpp-build/absl/synchronization/libabsl_synchronization.a
./_deps/abseil-cpp-build/absl/strings/libabsl_strings.a
./_deps/abseil-cpp-build/absl/strings/libabsl_str_format_internal.a
./_deps/abseil-cpp-build/absl/strings/libabsl_cord.a
./_deps/abseil-cpp-build/absl/strings/libabsl_strings_internal.a
./_deps/abseil-cpp-build/absl/status/libabsl_status.a
./_deps/abseil-cpp-build/absl/hash/libabsl_city.a
./_deps/abseil-cpp-build/absl/hash/libabsl_wyhash.a
./_deps/abseil-cpp-build/absl/hash/libabsl_hash.a
./_deps/abseil-cpp-build/absl/flags/libabsl_flags_reflection.a
./_deps/abseil-cpp-build/absl/flags/libabsl_flags_program_name.a
./_deps/abseil-cpp-build/absl/flags/libabsl_flags_internal.a
./_deps/abseil-cpp-build/absl/flags/libabsl_flags_private_handle_accessor.a
./_deps/abseil-cpp-build/absl/flags/libabsl_flags_marshalling.a
./_deps/abseil-cpp-build/absl/flags/libabsl_flags_commandlineflag_internal.a
./_deps/abseil-cpp-build/absl/flags/libabsl_flags_commandlineflag.a
./_deps/abseil-cpp-build/absl/flags/libabsl_flags_config.a
./_deps/abseil-cpp-build/absl/flags/libabsl_flags.a
./_deps/abseil-cpp-build/absl/numeric/libabsl_int128.a
./_deps/abseil-cpp-build/absl/debugging/libabsl_symbolize.a
./_deps/abseil-cpp-build/absl/debugging/libabsl_debugging_internal.a
./_deps/abseil-cpp-build/absl/debugging/libabsl_demangle_internal.a
./_deps/abseil-cpp-build/absl/debugging/libabsl_stacktrace.a
./_deps/abseil-cpp-build/absl/base/libabsl_spinlock_wait.a
./_deps/abseil-cpp-build/absl/base/libabsl_raw_logging_internal.a
./_deps/abseil-cpp-build/absl/base/libabsl_malloc_internal.a
./_deps/abseil-cpp-build/absl/base/libabsl_throw_delegate.a
./_deps/abseil-cpp-build/absl/base/libabsl_exponential_biased.a
./_deps/abseil-cpp-build/absl/base/libabsl_base.a
./_deps/abseil-cpp-build/absl/base/libabsl_log_severity.a
./_deps/abseil-cpp-build/absl/time/libabsl_time_zone.a
./_deps/abseil-cpp-build/absl/time/libabsl_civil_time.a
./_deps/abseil-cpp-build/absl/time/libabsl_time.a
./_deps/abseil-cpp-build/absl/container/libabsl_hashtablez_sampler.a
./_deps/abseil-cpp-build/absl/container/libabsl_raw_hash_set.a
./_deps/abseil-cpp-build/absl/types/libabsl_bad_variant_access.a
./_deps/abseil-cpp-build/absl/types/libabsl_bad_optional_access.a
./_deps/cpuinfo-build/libcpuinfo.a

The state of the libs seems to be the following:

absl: 30 libraries found
eigen: OK, template library defined in the headers
farmhash: OK, 1 library found
fft2d: OK, 2 libraries found
flatbuffers: OK, 1 library found
gemmlowp: OK, headers only
neon2sse: OK, headers only
clog: OK, 1 library found
cpuinfo: OK, 1 library found
ruy: 30 libraries found

All in all, most libs are OK, either there is 1 lib to link with, or the libs are header-only. What remains to be the problem are:

absl
ruy

because they contain about 30 .a libs. Not sure if I have to link with all of those? It would be very cumbersome, as my build system is Meson and I am using custom_target() to link with TensorFlow.

Bart · Answer 1 · 2022-08-17T08:31:44.623

A year later, but I just went through this myself, so here goes my answer.
Based on my experience (using a makefile and without your -DTFLITE_C_BUILD_SHARED_LIBS:BOOL=OFF) a program that performs inference does not need to link to Abseil.
You need to link to all other libs you mentioned, except ruy_kernel_arm and ruy_pack_arm, assuming you're running your program on the x64 platform. (Annoyingly -DTFLITE_ENABLE_RUY=OFF is not respected when building TfLite, so you're stuck with Ruy)

Detailed steps:

Build TfLite:

mkdir ~/my_tflite_project
cd ~/my_tflite_project/
git clone https://github.com/tensorflow/tensorflow.git tensorflow_src
mkdir tflite_build_x64
cd tflite_build_x64/
cmake ../tensorflow_src/tensorflow/lite/
    /* You may encounter two CMake messages:
    -- The Fortran compiler identification is unknown
    I believe a Fortran compiler is only necessary to build Fortran bindings for TfLite.

    -- Could NOT find CLANG_FORMAT: Found unsuitable version "0.0", but required is exact version "9" (found CLANG_FORMAT_EXECUTABLE-NOTFOUND)
    sudo apt install clang-format-9
    Annoyingly you need clang-format-9, plain clang-format (version 13, the newest) won't do. */
cmake --build . -j 4

Relevant libs:

$ cd ~/my_tflite_project/tflite_build_x64
$ ls *.a
libtensorflow-lite.a
$ ls pthreadpool/*.a
pthreadpool/libpthreadpool.a
$ ls _deps/*/*.a
_deps/clog-build/libclog.a        _deps/farmhash-build/libfarmhash.a  _deps/fft2d-build/libfft2d_fftsg2d.a      _deps/xnnpack-build/libXNNPACK.a
_deps/cpuinfo-build/libcpuinfo.a  _deps/fft2d-build/libfft2d_fftsg.a  _deps/flatbuffers-build/libflatbuffers.a
$ ls _deps/ruy-build/ruy/*.a
_deps/ruy-build/ruy/libruy_allocator.a         _deps/ruy-build/ruy/libruy_ctx.a                           _deps/ruy-build/ruy/libruy_kernel_avx.a       _deps/ruy-build/ruy/libruy_prepacked_cache.a
_deps/ruy-build/ruy/libruy_apply_multiplier.a  _deps/ruy-build/ruy/libruy_denormal.a                      _deps/ruy-build/ruy/libruy_kernel_avx2_fma.a  _deps/ruy-build/ruy/libruy_prepare_packed_matrices.a
_deps/ruy-build/ruy/libruy_block_map.a         _deps/ruy-build/ruy/libruy_frontend.a                      _deps/ruy-build/ruy/libruy_kernel_avx512.a    _deps/ruy-build/ruy/libruy_system_aligned_alloc.a
_deps/ruy-build/ruy/libruy_blocking_counter.a  _deps/ruy-build/ruy/libruy_have_built_path_for_avx.a       _deps/ruy-build/ruy/libruy_pack_arm.a         _deps/ruy-build/ruy/libruy_thread_pool.a
_deps/ruy-build/ruy/libruy_context.a           _deps/ruy-build/ruy/libruy_have_built_path_for_avx2_fma.a  _deps/ruy-build/ruy/libruy_pack_avx.a         _deps/ruy-build/ruy/libruy_trmul.a
_deps/ruy-build/ruy/libruy_context_get_ctx.a   _deps/ruy-build/ruy/libruy_have_built_path_for_avx512.a    _deps/ruy-build/ruy/libruy_pack_avx2_fma.a    _deps/ruy-build/ruy/libruy_tune.a
_deps/ruy-build/ruy/libruy_cpuinfo.a           _deps/ruy-build/ruy/libruy_kernel_arm.a                    _deps/ruy-build/ruy/libruy_pack_avx512.a      _deps/ruy-build/ruy/libruy_wait.a

Construct MWE using TfLite:

$ cd ~/my_tflite_project
$ mkdir my_dev_x64
$ cd my_dev_x64/
/* Construct minimal.cpp and makefile below */
$ cat minimal.cpp
#include "tensorflow/lite/model.h"
#include "tensorflow/lite/interpreter.h"
#include "tensorflow/lite/kernels/register.h"
#include <iostream>

int main() {
    std::unique_ptr<tflite::FlatBufferModel> model = tflite::FlatBufferModel::BuildFromFile("your_network_here.tflite");
    tflite::ops::builtin::BuiltinOpResolver resolver;

    std::cout "Done\n";
    return EXIT_SUCCESS;
}

$ cat makefile
COMPILER     := g++

LINKER       := g++

CXX_FILES    := minimal.cpp

OBJ_FILES    := $(CXX_FILES:.cpp=.o)

EXE_FILE     := app

INCLUDE_DIRS := -I../tensorflow_src -I../tflite_build_x64/flatbuffers/include

LIB_DIRS     := \
        -L../tflite_build_x64 \
        -L../tflite_build_x64/_deps/fft2d-build \
        -L../tflite_build_x64/_deps/flatbuffers-build \
        -L../tflite_build_x64/_deps/ruy-build/ruy \
        -L../tflite_build_x64/_deps/farmhash-build \
        -L../tflite_build_x64/_deps/xnnpack-build \
        -L../tflite_build_x64/_deps/cpuinfo-build \
        -L../tflite_build_x64/_deps/clog-build \
        -L../tflite_build_x64/pthreadpool

LIBS         := \
        -ltensorflow-lite \
        -lfft2d_fftsg \
        -lfft2d_fftsg2d \
        -lflatbuffers \
        -lruy_ctx \
        -lruy_allocator \
        -lruy_frontend \
        -lruy_context_get_ctx \
        -lruy_context \
        -lruy_apply_multiplier \
        -lruy_prepacked_cache \
        -lruy_tune \
        -lruy_cpuinfo \
        -lruy_system_aligned_alloc \
        -lruy_prepare_packed_matrices \
        -lruy_trmul \
        -lruy_block_map \
        -lruy_denormal \
        -lruy_thread_pool \
        -lruy_blocking_counter \
        -lruy_wait \
        -lruy_kernel_avx \
        -lruy_kernel_avx2_fma \
        -lruy_kernel_avx512 \
        -lruy_pack_avx \
        -lruy_pack_avx2_fma \
        -lruy_pack_avx512 \
        -lruy_have_built_path_for_avx \
        -lruy_have_built_path_for_avx2_fma \
        -lruy_have_built_path_for_avx512 \
        -lfarmhash \
        -lXNNPACK \
        -lpthreadpool \
        -lcpuinfo \
        -lclog

CXX_FLAGS    := -Wall #-pedantic

LINK_FLAGS   :=

#Do not print the output of the commands
.SILENT:

#Phony targets do not represent actual files, so files with the following names are ignored
.PHONY: clean depend

#Link object files to form an executable file
$(EXE_FILE): $(OBJ_FILES)
        $(LINKER) $(LINK_FLAGS) $(OBJ_FILES) -o $(EXE_FILE) $(LIB_DIRS) $(LIBS)

#Compile cpp files to object files
%.o: %.cpp
        $(COMPILER) $(CXX_FLAGS) $(INCLUDE_DIRS) -c $<

#Remove object files, executable, and possible linkinfo files
clean:
        -rm -f $(OBJ_FILES) $(EXE_FILE)

#Generate dependency file
depend:
        $(COMPILER) $(CXX_FLAGS) $(INCLUDE_DIRS) -MM $(CXX_FILES) > make.dep

#Include dependency file
-include make.dep

Build and run MWE:

$ cd ~/my_tflite_project/my_dev_x64
$ make
$ ./app
Done

Hopefully that helps you or some other poor fellow to get TfLite C++ working.

I even tried to link all `.a` into one `tf_dependencies.a` lib: https://stackoverflow.com/questions/71585671/failed-to-link-with-tensorflow-lite-c-api-static-library-undefined-reference-to. That didn't work too. — Danijel, May 04 '22 at 08:59
Here is a script on how to merge many libs into one: https://stackoverflow.com/questions/3821916/how-to-merge-two-ar-static-libraries-into-one/71585230#71585230. — Danijel, May 04 '22 at 09:02
Another thing: I must use TF Lite C API, not C++. And finally, I need to build for x8664, but also for Arm (armv7 32-bit and armv8 64-bit). — Danijel, May 04 '22 at 09:03
For Arm, replace `-lruy_kernel_avx*`, `-lruy_pack_avx*`, and `-lruy_have_built_path_for_avx*` with `-lruy_kernel_arm` and `-lruy_pack_arm`. Or always link to all the above libs if you don't mind redundancy on your platform. — Bart, May 05 '22 at 15:06
I have also been thinking about merging the static libs into one static lib. But I deferred this to the end. At the moment I have it working on x64 and Arm64. At the end of the project I probably have more insight in how I want to package the software. I cannot distill from your answer whether you have TfLite working yet. — Bart, May 05 '22 at 15:09
I only have experience with the C++ API. I would guess that with the C API you want to link to something like `-libtensorflowlite_c.a`. Do that, and then determine which other libraries you need. — Bart, May 05 '22 at 15:10
For now I am using the shared lib - meaning, I still haven't been able to use the static lib. I have built the static lib, but then I have zillion of dependencies which I don't know how to link into final app - that's what [this related question](https://stackoverflow.com/questions/71585671/failed-to-link-with-tensorflow-lite-c-api-static-library-undefined-reference-to) is about. — Danijel, May 05 '22 at 15:25
Expanded my answer with detailed steps. Hopefully that should get you a working example. Good luck. — Bart, May 09 '22 at 08:10
Could you run the app, and give us the output, so that we are sure app has everything it needs to link with to run? — Danijel, May 09 '22 at 09:42
Of course I could I run the compiled app. As for the output, there is no other output then "Done". :? It's a minimal-working example to test whether TfLite is working (since that is you're question about). To do inference you should extend the example. — Bart, May 09 '22 at 10:33

TensorFlow static C API library - how to link with 10 sub-dependencies?

1 Answers1

Detailed steps:

Linked