Deploying a TensorFlow model on Google Cloud that receives a base64 encoded string as a model input

Question

I have successfully setup Google Cloud and deployed a pre-trained ML model that takes an input tensor (image) of shape=(?, 224, 224, 3) and dtype=float32. It works well but this is inefficient when making REST requests and should really use a base64 encoded string. The challenge is that I am using transfer learning and cannot control the input of the original pre-trained model. To get around this with adding additional infrastructure I created a small graph (wrapper) that handles the base64 to array conversion and connected it to my pre-trained model graph yielding a new single graph. The small graph takes an input tensor with the shape=(), dtype=string and return a tensor with the shape=(224, 224, 3), dtype=float32 which can then be passed to the original model. The model compiles to .pb file without errors and successfully deploys but I get the following error when making my Post request:

{'error': 'Prediction failed: Error during model execution: AbortionError(code=StatusCode.INVALID_ARGUMENT, details="Index out of range using input dim 0; input has only 0 dims\n\t [[{{node lambda/map/while/strided_slice}}]]")'}

Post request body:

{'instances': [{'b64': 'iVBORw0KGgoAAAANSUhEUgAAAOAA...'}]}`

This error leads me to believe the post request is incorrectly formatted for handling the base64 string or my base conversion graph input is setup incorrectly. I can run the code locally by calling predict on my combined model and pass it a tensor in the form of shape=(), dtype=string constructed locally and get a result successfully.

Here is my code for combining the 2 graphs:

import tensorflow as tf

# Local dependencies
from myProject.classifier_models import mobilenet
from myProject.dataset_loader import dataset_loader
from myProject.utils import f1_m, recall_m, precision_m

with tf.keras.backend.get_session() as sess:    

    def preprocess_and_decode(img_str, new_shape=[224,224]):
        #img = tf.io.decode_base64(img_str)
        img = tf.image.decode_png(img_str, channels=3)
        img = (tf.cast(img, tf.float32)/127.5) - 1
        img = tf.image.resize_images(img, new_shape, method=tf.image.ResizeMethod.AREA, align_corners=False)
        # If you need to squeeze your input range to [0,1] or [-1,1] do it here
        return img

    InputLayer = tf.keras.layers.Input(shape = (1,),dtype="string")
    OutputLayer = tf.keras.layers.Lambda(lambda img : tf.map_fn(lambda im : preprocess_and_decode(im[0]), img, dtype="float32"))(InputLayer)
    base64_model = tf.keras.Model(InputLayer,OutputLayer)

    tf.keras.backend.set_learning_phase(0)  # Ignore dropout at inference
    transfer_model = tf.keras.models.load_model('./trained_model/mobilenet_93.h5', custom_objects={'f1_m': f1_m, 'recall_m': recall_m, 'precision_m': precision_m})
    sess.run(tf.global_variables_initializer())

    base64_input = base64_model.input
    final_output = transfer_model(base64_model.output)
    new_model = tf.keras.Model(base64_input,final_output)

    export_path = '../myModels/001'

    tf.saved_model.simple_save(
        sess,
        export_path,
        inputs={'input_class': new_model.input},
        outputs={'output_class': new_model.output})

Tech: TensorFlow 1.13.1 & Python 3.5

I have looked at a bunch of related posts such as:

https://stackoverflow.com/a/50606625

https://stackoverflow.com/a/42859733

http://www.voidcn.com/article/p-okpgbnul-bvs.html (right-click translate to english)

https://cloud.google.com/ml-engine/docs/tensorflow/online-predict

Any suggestions or feedback would be greatly appreciated!

Update 06/12/2019:

Inspecting the 3 graph summaries everything appears correctly merged