1

I am trying to figure out how to do batch prediction using Google Cloud. Specifically, I'm looking to do object detection, getting from a faster-RCNN tensorflow ckpt to a graph/saved model.

My issue is that I need to be able to recover some kind of ID for my input images, perhaps an index or a filename. I'm not entirely sure how to do this in my situation, since this link mentions using instance keys, and the only relevant examples I've found regarding instance keys use JSON as the input format. As I am supposed to use TFRecords for input to my saved model, this would seem to be an issue. I also consulted the prediction guide, but was still confused.

In short, does anybody have any tips as to what file(s) I should edit (export_inference_graph.py?) to preserve some sort of indices/ordering of my input images for batch prediction? I am using the Object Detection API for reference. Thanks!

Chris Collins
  • 13
  • 4

1 Answers1

0

Batch Prediction doesn't support instance keys by itself. You have to change the inference graph to output something from input as keys. That means you need to find a way to include keys in your inputs such as image id or index. One way you can do it is to change your input from TFrecord to json and add a id as keys. E.g, your input now would be like:

{"key": 1, "image": {"b64": "base64encodedstringabce"} {"key": 2, "image": {"b64": "base64encodedstringfg1d"}

This would of course make your input much larger. Another way is that if you use tf.Example proto in your TFRecord, you can add an extra feature. Its value would be a passthrough field from input to the output.

Here is a way to change the inference to pass a feature from input to output. https://github.com/GoogleCloudPlatform/cloudml-samples/pull/158

yxshi
  • 244
  • 1
  • 5
  • Thanks for the quick response. To clarify, judging by the "Predict the input" section of the guide [here](https://cloud.google.com/blog/big-data/2017/09/performing-prediction-with-tensorflow-object-detection-models-on-google-cloud-machine-learning-engine), I only have one input tensor to work with. Thus, it seems that I'd need to change the number of input tensors to two to handle either a json or tfrecord with a key; do you know how I'd do that for an object detection inference graph? – Chris Collins Jun 15 '18 at 15:42
  • Yes, you are right. Not sure what "object detection inference graph" means. Is it a string tensor? If so, just follow the way in my answer. Otherwise, you take a look at the PR shown above. – yxshi Jun 24 '18 at 17:34
  • The saved model/frozen graph the Object Detection API again only has a single (string?) input tensor, as specified [here](https://cloud.google.com/blog/big-data/2017/09/performing-prediction-with-tensorflow-object-detection-models-on-google-cloud-machine-learning-engine). Your example seems to involve a lot of csv stuff, which I'm not sure is relevant to what I'm doing. Basically, as it works right now, my model only accepts a batch of base64-encoded versions of my input images, and no other info (i.e., no keys); if you could explain how your solution could translate to this, I'd be thankful. – Chris Collins Jun 28 '18 at 16:33
  • I haven't tried by myself but I assume you can make changes in https://github.com/tensorflow/models/blob/master/research/object_detection/exporter.py#L135 just like the PR I mentioned. Basically you change the inference graph to add an extra "pass-through" tensor. – yxshi Jul 18 '18 at 21:58
  • 1
    OK, forgot to mention that I found a way to do it via somebody's notebook on github. Thanks a bunch for your help, though! – Chris Collins Jul 23 '18 at 15:42
  • 1
    @ChrisCollins It would be great if you could please post the solution as well for others stumbling across this problem? – Hassan Kamal Aug 10 '18 at 17:55
  • @ChrisCollins same request: could you please post your solution? – Patrick Nov 08 '20 at 15:05
  • Apologies all for the delayed reply -- I mainly reappropriated work from this script and had no issues https://github.com/priya-dwivedi/Deep-Learning/blob/master/Object_Detection_Tensorflow_API.ipynb – Chris Collins Nov 11 '20 at 03:52