Serving Predictions with Google Cloud Platform and AI Platform

Question

I have trained an AI model with the help of tensorflow and need to use Google AI Platform to serve predictions.

Now AI Platform specifies that the model needs to be in a specific 'SavedModel' format for me to upload the model to cloud and serve predictions.

How do I convert the model to the specified 'SavedModel' format?

Also, are there any end-to-end tutorials available that would help me do the same?

When you define your model, you have to define an exporter. This exporter is, most of time, a storage bucket. Did you have this ? — guillaume blaquiere, Sep 06 '19 at 11:41
What is your current model format? Take a look at this: https://stackoverflow.com/questions/44329185/convert-a-graph-proto-pb-pbtxt-to-a-savedmodel-for-use-in-tensorflow-serving-o/44329200#44329200 and https://www.tensorflow.org/guide/saved_model#build_and_load_a_savedmodel — gogasca, Sep 09 '19 at 19:03
if you need more specific help, please post the way you are saving it now. — Lak, Sep 12 '19 at 05:53
Can I ask you @guillaumeblaquiere where you define such an exporter? In the pipeline config file? — Patrick, Nov 03 '20 at 22:13
With tensorflow, you have it in your training loop. But you don't mention that you use tensorflow. And why are you talking about pipeline? Can you precise your question/context, with code sample? — guillaume blaquiere, Nov 04 '20 at 08:17
I launch my training command with a command like 'gcloud ai-platform submit training ... model_main_tf2.py ...'. You mean that I should update the model_main_tf2 script to add an exporter? — Patrick, Nov 04 '20 at 20:40
And when I said pipeline configs, I was referring to configs needed for training such as those found here: https://github.com/tensorflow/models/tree/master/research/object_detection/samples/configs — Patrick, Nov 04 '20 at 20:46
@guillaumeblaquiere I'm using tensorflow object-detection API with Google AI Platform. — Patrick, Nov 04 '20 at 22:49
I'm not a data scientist and all that I know, and test, I learnt all what I know on Tensorflow from @Lak and his great coursera courses!! Anyway, I submitted an answer, simply because it's easier to explain. It's not the exporter, but the estimator that export the model. — guillaume blaquiere, Nov 05 '20 at 08:26

score 1 · Answer 1 · answered Nov 05 '20 at 08:23

In a standard training loop, you should have code like this at the end

.....
def train_and_evaluate(output_dir, hparams):
    get_train = read_dataset(hparams['train_data_path'],
                             tf.estimator.ModeKeys.TRAIN,
                             hparams['train_batch_size'])
    get_valid = read_dataset(hparams['eval_data_path'],
                             tf.estimator.ModeKeys.EVAL,
                             1000)
    estimator = tf.estimator.Estimator(model_fn=sequence_regressor,
                                       params=hparams,
                                       config=tf.estimator.RunConfig(
                                           save_checkpoints_steps=
                                           hparams['save_checkpoint_steps']),
                                       model_dir=output_dir)
    train_spec = tf.estimator.TrainSpec(input_fn=get_train,
                                        max_steps=hparams['train_steps'])
    exporter = tf.estimator.LatestExporter('exporter', serving_input_fn)
    eval_spec = tf.estimator.EvalSpec(input_fn=get_valid,
                                      steps=None,
                                      exporters=exporter,
                                      start_delay_secs=hparams['eval_delay_secs'],
                                      throttle_secs=hparams['min_eval_frequency'])
    tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)

especially this part

estimator = tf.estimator.Estimator(model_fn=sequence_regressor,
                                       params=hparams,
                                       config=tf.estimator.RunConfig(
                                           save_checkpoints_steps=
                                           hparams['save_checkpoint_steps']),
                                       model_dir=output_dir)

Here you specify after how many steps (save_checkpoints_steps) you export your model to the output_dir.

Do you have something like this in your code?

Serving Predictions with Google Cloud Platform and AI Platform

1 Answers1