3

I want to run a tflite model on an ARM Linux board. here are the steps I followed:

1-Converted my Tensorflow 2 linear estimator model to tflite with select ops using python API:

import tensorflow as tf
loaded = tf.saved_model.load(saved_model_path, tags = None)
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir = saved_model_dir, signature_keys=['serving_default'])
converter.experimental_new_converter = True
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]
tflite_model = converter.convert()
with open('model.tflite', 'wb') as f:
  f.write(tflite_model)

2-Added "//tensorflow/lite/delegates/flex:delegate" to deps[] of tensorflowlite in /tensorflow/lite/Build for dependencies as it is mentioned here.

3-Built the tflite shared library on my pc using following command:

bazel-3.1.0 build --config=elinux_armhf --config=monolithic -c opt --config=v2 //tensorflow/lite/c:libtensorflowlite_c.so

4-Renamed output shared library located at bazel-bin/tensorflow/lite/libtensorflowlite_c.so to libtensorflow-lite.so

5-Copied model.tflite, demo.cc and libtensorflow-lite.so to my device at /opt/demo and the demo.cc content is:

#include <stdio.h>
#include "tensorflow/lite/interpreter.h"
#include "tensorflow/lite/kernels/register.h"
#include "tensorflow/lite/model.h"
#include "tensorflow/lite/tools/gen_op_registration.h"

int main(){

    std::unique_ptr<tflite::FlatBufferModel> model = tflite::FlatBufferModel::BuildFromFile("model.tflite");

    if(!model){
        printf("Failed to mmap model\n");
        exit(0);
    }

    tflite::ops::builtin::BuiltinOpResolver resolver;
    std::unique_ptr<tflite::Interpreter> interpreter;
    tflite::InterpreterBuilder(*model.get(), resolver)(&interpreter);

    // Resize input tensors, if desired.
    interpreter->AllocateTensors();

    float* input = interpreter->typed_input_tensor<float>(0);
    // Dummy input for testing
    *input = 2.0;

    interpreter->Invoke();

    float* output = interpreter->typed_output_tensor<float>(0);

    printf("Result is: %f\n", *output);

    return 0;
}

6-copied the Tensorflow source to /opt of my ARM board.

7-Created the object file by running following command on my device:

/opt/demo# g++ demo.cc -c -I /opt/tensorflow_src -I /opt/tensorflow_src/flatbuffers/

8-When I run g++ -L/opt/demo/ -o exec demo.o -ltensorflow-lite command to generate binaries I get following errors:

/usr/lib/gcc/arm-poky-linux-gnueabi/9.3.0/../../../../arm-poky-linux-gnueabi/bin/ld: demo.o: in function `main':
demo.cc:(.text+0xc): undefined reference to `tflite::DefaultErrorReporter()'
/usr/lib/gcc/arm-poky-linux-gnueabi/9.3.0/../../../../arm-poky-linux-gnueabi/bin/ld: demo.cc:(.text+0x24): undefined reference to `tflite::FlatBufferModel::BuildFromFile(char const*, tflite::ErrorReporter*)'
/usr/lib/gcc/arm-poky-linux-gnueabi/9.3.0/../../../../arm-poky-linux-gnueabi/bin/ld: demo.cc:(.text+0x68): undefined reference to `tflite::ops::builtin::BuiltinOpResolver::BuiltinOpResolver()'
/usr/lib/gcc/arm-poky-linux-gnueabi/9.3.0/../../../../arm-poky-linux-gnueabi/bin/ld: demo.cc:(.text+0x90): undefined reference to `tflite::InterpreterBuilder::InterpreterBuilder(tflite::FlatBufferModel const&, tflite::OpResolver const&)'
/usr/lib/gcc/arm-poky-linux-gnueabi/9.3.0/../../../../arm-poky-linux-gnueabi/bin/ld: demo.cc:(.text+0xa4): undefined reference to `tflite::InterpreterBuilder::operator()(std::unique_ptr<tflite::Interpreter, std::default_delete<tflite::Interpreter> >*)'
/usr/lib/gcc/arm-poky-linux-gnueabi/9.3.0/../../../../arm-poky-linux-gnueabi/bin/ld: demo.cc:(.text+0xb0): undefined reference to `tflite::InterpreterBuilder::~InterpreterBuilder()'
/usr/lib/gcc/arm-poky-linux-gnueabi/9.3.0/../../../../arm-poky-linux-gnueabi/bin/ld: demo.cc:(.text+0xc8): undefined reference to `tflite::Interpreter::AllocateTensors()'
/usr/lib/gcc/arm-poky-linux-gnueabi/9.3.0/../../../../arm-poky-linux-gnueabi/bin/ld: demo.cc:(.text+0x110): undefined reference to `tflite::Interpreter::Invoke()'
/usr/lib/gcc/arm-poky-linux-gnueabi/9.3.0/../../../../arm-poky-linux-gnueabi/bin/ld: demo.cc:(.text+0x190): undefined reference to `tflite::InterpreterBuilder::~InterpreterBuilder()'
/usr/lib/gcc/arm-poky-linux-gnueabi/9.3.0/../../../../arm-poky-linux-gnueabi/bin/ld: demo.o: in function `tflite::MutableOpResolver::~MutableOpResolver()':
demo.cc:(.text._ZN6tflite17MutableOpResolverD2Ev[_ZN6tflite17MutableOpResolverD5Ev]+0x6c): undefined reference to `vtable for tflite::MutableOpResolver'
/usr/lib/gcc/arm-poky-linux-gnueabi/9.3.0/../../../../arm-poky-linux-gnueabi/bin/ld: demo.o: in function `tflite::ops::builtin::BuiltinOpResolver::~BuiltinOpResolver()':
demo.cc:(.text._ZN6tflite3ops7builtin17BuiltinOpResolverD2Ev[_ZN6tflite3ops7builtin17BuiltinOpResolverD5Ev]+0x4c): undefined reference to `vtable for tflite::ops::builtin::BuiltinOpResolver'
/usr/lib/gcc/arm-poky-linux-gnueabi/9.3.0/../../../../arm-poky-linux-gnueabi/bin/ld: demo.o: in function `std::default_delete<tflite::FlatBufferModel>::operator()(tflite::FlatBufferModel*) const':
demo.cc:(.text._ZNKSt14default_deleteIN6tflite15FlatBufferModelEEclEPS1_[_ZNKSt14default_deleteIN6tflite15FlatBufferModelEEclEPS1_]+0x24): undefined reference to `tflite::FlatBufferModel::~FlatBufferModel()'
/usr/lib/gcc/arm-poky-linux-gnueabi/9.3.0/../../../../arm-poky-linux-gnueabi/bin/ld: demo.o: in function `std::default_delete<tflite::Interpreter>::operator()(tflite::Interpreter*) const':
demo.cc:(.text._ZNKSt14default_deleteIN6tflite11InterpreterEEclEPS1_[_ZNKSt14default_deleteIN6tflite11InterpreterEEclEPS1_]+0x24): undefined reference to `tflite::Interpreter::~Interpreter()'
collect2: error: ld returned 1 exit status

How can I resolve this issue and compile my interpreter binaries?

Filthy Man
  • 71
  • 1
  • 11

1 Answers1

1

I rechecked my steps and noticed a problem in my bazel build command, I was building the .so lib for C instead of C++. I was able to solve the problem by building libtensorflow for C++ using following command:

bazel-3.1.0 build --config=elinux_armhf --config=monolithic -c opt //tensorflow/lite:libtensorflowlite.so --local_ram_resources=10240 --config=noaws

At first, I tried to build without noaws flag but since during the dependency download stage of bazel, an aws related package link was broken the build failed, However I resolved the issue by adding noaws flag.

This post can be useful for people who are trying to convert a Tensorflow2 linear estimator model to a TensorFlow lite model and want to build the C++ libs with select TensorFlow ops for ARM architecture using Bazel.

Filthy Man
  • 71
  • 1
  • 11