0

I have this code for building a semantic search engine using pre-trained universal encoder from tensorflow hub. I am not able to convert to tlite. I have saved the model to my directory.

Importing the model:

module_path ="/content/drive/My Drive/4"
%time model = hub.load(module_path)
#print ("module %s loaded" % module_url)

#Create function for using modeltraining
def embed(input):
    return model(input)

Training the model on data:

## training the model
Model_USE= embed(data)

Saving the model:

exported = tf.train.Checkpoint(v=tf.Variable(Model_USE))
exported.f = tf.function(
    lambda  x: exported.v * x,
    input_signature=[tf.TensorSpec(shape=None, dtype=tf.float32)])
export_dir = "/content/drive/My Drive/"
tf.saved_model.save(exported,export_dir)

Saving works fine but when I convert to tflite it gives error.

Conversion code:

converter = tf.lite.TFLiteConverter.from_saved_model(export_dir)
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS,
                                       tf.lite.OpsSet.SELECT_TF_OPS]
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)

Error: 

as_list() is not defined on an unknown TensorShape.
Ali Memon
  • 115
  • 2
  • 8
  • Hi @AlI I am trying to do the same. Have you been able to execute this model? If you have any sample for the same or pointer please point me to that Thanks a ton. – CodeJunkie May 02 '22 at 10:34

1 Answers1

1

First, you should need to add a data generator to have representative inputs for the converter. Just like this:

def representative_data_gen():
  for input_value in dataset.take(100):
    yield [input_value]

The input value must be of shape (1, your_iput_shape) as if it had batch shape of 1. It has to be yielded as a list; mandatory.

You should also declare which type of optimization do you want, for example:

converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]

Nevertheless, I have also encountered problems with the different options of the converter depending on the network structure, which in this case I do not know. So, to make a clean run of the converter I would just do:

converter = lite.TFLiteConverter.from_keras_model(model)
converter.experimental_new_converter = True
converter.optimizations = [lite.Optimize.DEFAULT]
tfmodel = converter.convert()

The converter.experimental_new_converter = True is for problems when converting RNNs as in https://github.com/tensorflow/tensorflow/issues/34813

EDIT:

As explained here: ValueError: None is only supported in the 1st dimension. Tensor 'flatbuffer_data' has invalid shape '[None, None, 1, 512]' TFLite only allows the first dimension of your data to be None, that is, the batch. All other dimensions must be fixed. Try padding them with, for example, tf.keras.preprocessing.sequence.pad_sequences.

Then mask your sequences in the network as described in: tensorflow.org/guide/keras/masking_and_padding with Embedding or Masking layers.

BCJuan
  • 805
  • 8
  • 17
  • i don't know the shape of input. All I know is that it is text and can be any shape. – Ali Memon Apr 04 '20 at 09:19
  • As explained here: https://stackoverflow.com/questions/60826779/valueerror-none-is-only-supported-in-the-1st-dimension-tensor-flatbuffer-data TFLite only allows the first dimension of your data to be None, that is, the batch. All other dimensions must be fixed. Try padding them with, for example, `tf.keras.preprocessing.sequence.pad_sequences`. Then mask your sequences in the network as described in: https://www.tensorflow.org/guide/keras/masking_and_padding with `Embedding` or `Masking` layers. Adding this to the answer. – BCJuan Apr 04 '20 at 10:18