How to implement TensorFlow Serving Input function for images as base64-encoded strings and get prediction on Cloud ML Engine
I am planning to deploy the model on Cloud Machine Learning (ML) Engine after training it on-prem., but I have no idea how to implement the serving input function.
Additionally, I have tried to avoid TensorFlow low-level APIs and only focused on TensorFlow high-level APIs (TensorFlow Estimator). Below here in the blocks of code is the example code that I am working on.
import numpy as np
import tensorflow as tf
import datetime
import os
# create model
from tensorflow.python.keras.applications.vgg16 import VGG16
from tensorflow.python.keras import models
from tensorflow.python.keras import layers
conv_base = VGG16(weights='imagenet',
include_top=False,
input_shape=(150, 150, 3))
model = models.Sequential()
model.add(conv_base)
model.add(layers.Flatten())
model.add(layers.Dense(256, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
conv_base.trainable = False
model.compile(loss='binary_crossentropy',
optimizer=tf.keras.optimizers.RMSprop(lr=2e-5),
metrics=['acc'])
dt = datetime.datetime.now()
datetime_now = dt.strftime("%y%m%d_%H%M%S")
model_dir = 'models/imageclassifier_'+datetime_now
model_dir = os.path.join(os.getcwd(), model_dir)
if not os.path.exists(model_dir):
os.makedirs(model_dir)
print ("model_dir: ",model_dir)
est_imageclassifier = tf.keras.estimator.model_to_estimator(keras_model=model, model_dir=model_dir)
# input layer name
input_name = model.input_names[0]
input_name
This section is for image input function.
def imgs_input_fn(filenames, labels=None, perform_shuffle=False, repeat_count=1, batch_size=1):
def _parse_function(filename, label):
image_string = tf.read_file(filename)
image = tf.image.decode_image(image_string, channels=3)
image.set_shape([None, None, None])
image = tf.image.resize_images(image, [150, 150])
image = tf.subtract(image, 116.779) # Zero-center by mean pixel
image.set_shape([150, 150, 3])
image = tf.reverse(image, axis=[2]) # 'RGB'->'BGR'
d = dict(zip([input_name], [image])), label
return d
if labels is None:
labels = [0]*len(filenames)
labels=np.array(labels)
# Expand the shape of "labels" if necessary
if len(labels.shape) == 1:
labels = np.expand_dims(labels, axis=1)
filenames = tf.constant(filenames)
labels = tf.constant(labels)
labels = tf.cast(labels, tf.float32)
dataset = tf.data.Dataset.from_tensor_slices((filenames, labels))
dataset = dataset.map(_parse_function)
if perform_shuffle:
# Randomizes input using a window of 256 elements (read into memory)
dataset = dataset.shuffle(buffer_size=256)
dataset = dataset.repeat(repeat_count) # Repeats dataset this # times
dataset = dataset.batch(batch_size) # Batch size to use
iterator = dataset.make_one_shot_iterator()
batch_features, batch_labels = iterator.get_next()
return batch_features, batch_labels
I would like to create a serving input function that
Get images as base64-encoded strings in JSON format
Convert them into Tensors and reduce the size to (?, 150, 150, 3) for prediction
As shown below,
def serving_input_receiver_fn():
''' CODE HERE!'''
return tf.estimator.export.ServingInputReceiver(feature_placeholders, feature_placeholders)
To train and evaluate the model,
train_spec = tf.estimator.TrainSpec(input_fn=lambda: imgs_input_fn(train_files,
labels=train_labels,
perform_shuffle=True,
repeat_count=1,
batch_size=20),
max_steps=500)
exporter = tf.estimator.LatestExporter('Servo', serving_input_receiver_fn)
eval_spec = tf.estimator.EvalSpec(input_fn=lambda: imgs_input_fn(val_files,
labels=val_labels,
perform_shuffle=False,
batch_size=1),
exporters=exporter)
tf.estimator.train_and_evaluate(est_imageclassifier, train_spec, eval_spec)
If I understand it correctly, the example of the input file to get the prediction on Cloud ML Engine should be something like
request.json
{"b64": "9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHJC...”}
{"b64": "9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHJC...”}
And
gcloud ml-engine predict --model MODEL_NAME \
--version MODEL_VERSION \
--json-instances request.json
If you are reading until here and have some idea, could you please suggest me how to implement the serving input function for this particular case.
Many thanks in advance,
2nd Post - To update what I have done so far.
According to sdcbr's comment, below here is my serving_input_receiver_fn().
For _img_string_to_tensor() function or (prepare_image function), I guess that I should do image preparation the same way as I trained the model that you can see
imgs_input_fn() => _parse_function().
def serving_input_receiver_fn():
def _img_string_to_tensor(image_string):
image = tf.image.decode_image(image_string, channels=3)
image.set_shape([None, None, None])
image = tf.image.resize_images(image, [150, 150])
image = tf.subtract(image, 116.779) # Zero-center by mean pixel
image.set_shape([150, 150, 3])
image = tf.reverse(image, axis=[2]) # 'RGB'->'BGR'
return image
input_ph = tf.placeholder(tf.string, shape=[None])
images_tensor = tf.map_fn(_img_string_to_tensor, input_ph, back_prop=False, dtype=tf.float32)
return tf.estimator.export.ServingInputReceiver({model.input_names[0]: images_tensor}, {'image_bytes': input_ph})
After I trained the model and deployed the saved model on Cloud ML Engine. My input image was prepared into the format shown below.
{"image_bytes": {"b64": "YQ=="}}
But I found the error after getting a prediction via gcloud.
gcloud ml-engine predict --model model_1 \
--version v1 \
--json-instances request.json
{ "error": "Prediction failed: Error during model execution: AbortionError(code=StatusCode.INVALID_ARGUMENT, details=\"assertion failed: [Unable to decode bytes as JPEG, PNG, GIF, or BMP]\n\t [[{{node map/while/decode_image/cond_jpeg/cond_png/cond_gif/Assert_1/Assert}} = Assert[T=[DT_STRING], summarize=3, _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](map/while/decode_image/cond_jpeg/cond_png/cond_gif/is_bmp, map/while/decode_image/cond_jpeg/cond_png/cond_gif/Assert_1/Assert/data_0)]]\")" }
Did I do something wrong in _img_string_to_tensor function?
and Could you please clarify me more about this tf.placeholder?
input_ph = tf.placeholder(tf.string, shape=[None])
For your above code, you use shape=[1], but I think it should be shape=[None].
