GCP ML Engine Prediction failed: Error processing input: Expected float32 got base64

Question

I am trying to call a prediction on a custom trained TensorFlow model deployed to GCP ML engine. When I am trying to call a prediction on the model it is returning the following error message "Expected float32 got base64"

1. I've used transfer learning and the TensorFlow's retrain.py script to train my model on my images, following the official documentation

python retrain.py --image_dir ~/training_images saved_model_dir /saved_model_directory

2. I've tested the prediction locally using TensorFlow's label_img.py script, the prediction worked locally for my images

python label_image.py --graph=/tmp/output_graph.pb --labels=/tmp/output_labels.txt --input_layer=Placeholder --output_layer=final_result \

3. I've exported my model to use with Tensorflow Serving as described in the documentation of the retrain.py script.

python retrain.py --image_dir ~/training_images --saved_model_dir /saved_model_directory

4. I've uploaded the model to Firebase, GCP validated and accepted my model, I was able to trigger my model.

5. When trying to call an online prediction I am receiving the " Expected float32" error.

 test.json ={"image_bytes": {"b64": "/9j/4AAQSkZJ.......=="}}

 gcloud ml-engine predict \
        --model my_model \
        --version v1 \
        --json-instances ./test.json

Do I need to modify retrain.py to make my saved model accept base64 or is there any other solution for the problem?

I've already checked the following answer, but unfortunately it does not solved my problem: How to pass base64 encoded image to Tensorflow prediction?

Answer 1

The problem is that retrain.py exports a model whose input is expecting an already decoded and resized image in the form of floats (see this line), but you are passing it raw, undecoded image data.

There are two solutions.

1. Create a JSON request in the expected format (floats). This is an easy fix but could have performance implications (sending float32 data as JSON can be inefficient).
2. Alter the model to accept raw image data as input. This requires some reworking of the model.

For (1), you would send a JSON file similar to:

{"images": [[[0.0, 0.0, 0.0], [0,0,0], [...]], [...], ...]}

Of course, you'd probably construct that using some client library

(2) is a little more involved. This sample can guide you on how to do that.

Answer 2

While sending a float32 array in json will work, you'll find that it is extremely slow due to network latency. If at all possible, you'd want to use base64 encoded strings.

To do this, one very simple change you can make to the export script is to change the image input name to end with _bytes. When this occurs, tensorflow serving will decode your base64 encoded image string automatically for you. Essentially update this line https://github.com/tensorflow/hub/blob/a96bbd73abbecfad8c5517684cf3655b48bab39b/examples/image_retraining/retrain.py#L963

to inputs={'image_bytes': in_image},

This is probably one of the most useful but lesser known behavior of tensorflow serving when dealing with image inputs.

You can also use tf.io.decode and write your own decoding function when exporting the model.

Finally, your json payload would look something like this

{"inputs": {"image_bytes": {"b64": "/9j/4AAQSkZJ.......=="}}}

Or, if you prefer the instances format

{"instances": [{"image_bytes": {"b64": "/9j/4AAQSkZJ.......=="}}]}