2

I have a multiclass(108 classes) classification model which I want to apply transfer learning to the classification layer. I want to deploy this model in a low computing resource device (Raspberry Pi) and I thought to implement the classification layer in pure numpy instead of using Keras or TF. Below is my original model.

from tensorflow.keras.models import Sequential, Model, LSTM, Embedding
model = Sequential()
model.add(Embedding(108, 50, input_length=10))
model.add((LSTM(32, return_sequences=False)))
model.add(Dense(108, activation="softmax"))

model.compile(loss="categorical_crossentropy", optimizer=Adam(lr=0.001), metrics=['accuracy'])
model.summary()
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=3)
history = model.fit(X_train, y_train, epochs=10, batch_size=32, validation_split=0.5, callbacks=[es]).history

I split this model into two parts, encoder and decoder as follows. decoder is the classification layer which I want to convert into NumPy model and then do the on-device transfer learning later.

encoder = Sequential([
    Embedding(108, 50, input_length=10),
    GRU(32, return_sequences=False)
])

decoder = Sequential([
    Dense(108, activation="softmax")
])

model = Model(inputs=encoder.input, outputs=decoder(encoder.output))

model.compile(loss="categorical_crossentropy", optimizer=Adam(lr=0.001), metrics=['accuracy'])
model.summary()
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=3)
history = model.fit(X_train, y_train, epochs=50, batch_size=32, validation_split=0.5, callbacks=[es]).history

I have a few questions related to this approach.

  1. Only way i know to train this model is, first to train the encoder and decoder. then train NumPy classification layer using trained encoder outputs. Is there any way i can train the NumPy model at the same time when i train the encoder (without using the above Keras decoder part and Model)? I can't use Model as I can't use Keras or TF in raspberry Pi during the transfer learning.

  2. If there is no any way to train encoder and Numpy model at the same time, How to use learned decoder weights as the starting weights of the Numpy Model instead of starting from random weights?

  3. What is the most efficient code (or way) to implement the Numpy classification layer (decoder)? It requires a highly efficient model as i do the transfer learning on Raspberry Pi for incoming streaming data. Once i trained the model for reasonable data, i plan to convert the encoder into TFLite and do the inference

Highly appreciate any help or guidance to achieve this as I'm new to NumPy-based NN implementations. Thanks in advance

Sampath Rajapaksha
  • 111
  • 1
  • 1
  • 11
  • 1
    Why not train the entire network in Keras, then [convert the weights of the decoder to NumPy](https://stackoverflow.com/a/44938317/5231110), then continue training them on the Raspberry Pi? – root Dec 29 '22 at 12:44
  • 1
    Since you want to deploy the encoder with TFLite, it might be easiest to also continue training the decoder in TFLite as well. Regarding on-device training: https://www.tensorflow.org/lite/examples/on_device_training/overview – root Dec 29 '22 at 12:46
  • @root Thank you very much. Yes, i tried retraining with TFlite. But the issue is, this still requires installing TF in Raspberry PI (due to enabled TF operations) and more importantly inference of this model is bit slower than the normal TFlite model. I'm not sure why is this. I compared the TFlite inference of this model and On-device training model (above link) and, normal TFLite model is 10 times faster than the inference on above model which use on-device training. That's why i thought to use Numpy classification layer – Sampath Rajapaksha Dec 29 '22 at 13:03
  • As far as I understand, there are several options and you can compare which one better suits your needs: slow inference using the TFLite-trained model; convert (on the device) the TFLite-trained model to a normal TFLite model; convert (on the device) the TFLite-trained model to NumPy or to something else (are you asking how to convert it and what to convert it to?). – root Dec 29 '22 at 13:39
  • By the way, the TFLite-trained model having 10 times slower inference than a normal TFLite model seems like a bug in your code (maybe you forgot to switch to inference mode, i.e. switch off backprop and dropout) or in TFLite. – root Dec 29 '22 at 13:44
  • Thanks @root. Usually normal TFLite model takes around 0.0004s for the inference whereas the TFLite-trained model (on-device) takes around 0.004s for the inference under inference mode. I thought this is because usage of TF operations in on-device training model. convert (on the device) the TFLite-trained model to a normal TFLite model is a good Idea. I'll try a way to do it – Sampath Rajapaksha Dec 29 '22 at 13:54
  • 1
    Let me know what you find. Maybe ask on the TF forum whether that conversion is the intended usage, and how to implement it "officially". – root Dec 29 '22 at 14:14

0 Answers0