1

I am trying to take a simple keras model with an Add operation and convert to TFLite and then to EdgeTPU. Quantization for int8 needs to take place, but depending on the conversion parameters provided it results in either an unsupported operation FlexAddV2, or unsupported data type int32, or an error with AddV2 Error code: ERROR_NEEDS_FLEX_OPS.

The model and conversion are relatively simple and straightforward:

from tensorflow import keras
import numpy as np
import random

def representative_dataset():
    for _ in range(100):
        #data = random.randint(0, 1)
        #yield [data]
        data = np.random.rand(32)*2
        yield [data.astype(np.int8)]

input = keras.Input(shape=(32,), name="dummy_input", dtype=tf.int8)
output = tf.add(input, 1)
model = keras.Model(inputs=input, outputs=output)
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset
converter.target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS_INT8, # enable TensorFlow Lite ops.
tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.
tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.
]
converter.target_spec.supported_types = [tf.int8]
converter.inference_input_type = tf.int8 # or tf.uint8
converter.inference_output_type = tf.int8 # or tf.uint8
converter.experimental_new_quantizer = True # It will enable conversion and quantization of MLIR ops
converter.experimental_new_converter = False
tflite_quant_model = converter.convert()

Output from running the conversion:

Traceback (most recent call last):
File "/home/gsosnow/doc/gt2tf.py", line 27, in
tflite_quant_model = converter.convert()
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/lite.py", line 929, in wrapper
return self._convert_and_export_metrics(convert_func, *args, **kwargs)
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/lite.py", line 908, in _convert_and_export_metrics
result = convert_func(self, *args, **kwargs)
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/lite.py", line 1338, in convert
saved_model_convert_result = self._convert_as_saved_model()
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/lite.py", line 1320, in _convert_as_saved_model
return super(TFLiteKerasModelConverterV2,
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/lite.py", line 1131, in convert
result = _convert_graphdef(
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/convert_phase.py", line 212, in wrapper
raise converter_error from None # Re-throws the exception.
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/convert_phase.py", line 205, in wrapper
return func(*args, **kwargs)
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/convert.py", line 794, in convert_graphdef
data = convert(
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/convert.py", line 311, in convert
raise converter_error
tensorflow.lite.python.convert_phase.ConverterError: /home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/python/saved_model/save.py:1325:0: error: 'tf.AddV2' op is neither a custom op nor a flex op
:0: note: loc(fused["PartitionedCall:", "PartitionedCall"]): called from
/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/python/saved_model/save.py:1325:0: note: Error code: ERROR_NEEDS_FLEX_OPS
:0: error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: AddV2
Details:
tf.AddV2(tensor<?x32xi8>, tensor) -> (tensor<?x32xi8>) : {device = ""}
gsirocco
  • 11
  • 1
  • 1
  • You can try using the latest nightly build of TensorFlow to convert the model. Run `pip install tf-nightly` and then use `TFLiteConverter` from the installed nightly build of TF. – Shubham Panchal Sep 09 '22 at 02:39

1 Answers1

0

This was resolved here: https://github.com/google-coral/edgetpu/issues/655

Here is the python conversion code to accomplish this:

import tensorflow as tf
from tensorflow import keras
import numpy as np
import random

def representative_dataset():
  for _ in range(100):
      #data = random.randint(0, 1)
      #yield [data]
      data = np.random.rand(32)*2
      yield [data.astype(np.float32)]

input = keras.Input(shape=(32,), name="dummy_input", dtype=tf.float32)
output = tf.add(input, 1)
# output = tf.keras.layers.Add()([input, input])
model = keras.Model(inputs=input, outputs=output)
print(model.summary())
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.target_spec.supported_types = [tf.int8]
converter.inference_input_type = tf.int8 # or tf.uint8 
converter.inference_output_type = tf.int8 # or tf.uint8 

tflite_quant_model = converter.convert()
gsirocco
  • 11
  • 1
  • 1