1

I want to make online object detection predictions (or inference) from an already existing model in Google ML engine. But i am not able to build the json request.

The model is faster_rcnn_inception_resnet_v2_atrous_coco_2017_11_08 from the TF model zoo. The inputs are images the outputs class, bb, score etc...

The needed inputs are (from saved_model_cli show )

inputs['inputs'] tensor_info:
dtype: DT_UINT8
shape: (-1, -1, -1, 3)
name: image_tensor:0

As it expects an uint8 array I load the image into a numpy array 

encoded_contents = np.array(image.getdata()).reshape(
        (im_height, im_width, 3)).astype(np.uint8)

Resize the image image_np_expanded = np.expand_dims(encoded_contents, axis=0)

Tried to build the json request 

instance = {"input":encoded_contents}
row = json.dumps(instance,sort_keys=True)

But I am not able to build it because 

TypeError(repr(o) + " is not JSON serializable")
TypeError: array([[[164, 191, 220],
[190, 157, 114],
[190, 157, 114]]], dtype=uint8) is not JSON serializable

If I convert the numpy array to a list with the tolist() method the json file takes 3 megas and the ML-engine refuses it "message": "Request payload size exceeds the limit: 1572864 bytes.",

I will send this json to ml-engine predict as a json file.

gcloud ml-engine predict --model=pellaires --version=pellaires14 --json- 
instances=request.json > response.yaml
Juan Luis
  • 11
  • 3

2 Answers2

0

Sending an array of ints that big is not generally efficient (you'll spend non-trivial time encoding and decoding those arrays, latency on the network, etc.). This post provides a few options (including the tolist you tried).

I would recommend either "Tensors Packed as Byte Strings" or "Compressed Image Data".

UPDATE 10/19/2018

In order to use these approaches, you will need to modify your graph. If you're able to re-export the model, that is simplest. Instead of:

images = tf.placeholder(dtype=tf.uint8, shape=[None, None, None, 3])

You would use:

raw_byte_strings = tf.placeholder(dtype=tf.string, shape=[None])
decode = lambda raw_byte_str: tf.decode_raw(raw_byte_str, tf.uint8)
images = tf.map_fn(decode, raw_byte_strings, dtype=tf.uint8)
rhaertel80
  • 8,254
  • 1
  • 31
  • 47
  • Thank you. I already read that post but I think none of this inputs are accepted by my model that is expecting arrays dtype: DT_UINT8. Binary Image sizes are not so big, not more than 100kB it gets 3Mb when I use the tolist() method for building the json (as the array is not serialziable) – Juan Luis Oct 17 '18 at 18:05
  • Thank you. I finally moved to tensorflow_serving API serving it in kubernetes. Much more flexible and debugable. In ML API this was an error unsolvable as it made no sense (I was following Google published howto's). Moreover my clients python code is deployed to app engine (and I was considering use android mobile phones) so I wanted them as lighter as possible. So, I didn't wanted to load tensorflow in my client (the ones that sends the images to inference) and just numpy and python (not th.placeholder) – Juan Luis Nov 07 '18 at 18:03
0

The images I am working with are normalized to [0,1]. When converted to a list, each pixel of the image has an unnecessarily large amount of precision:

[[[0.4, 0.41568627450980394, 0.4117647058823529],
  [0.39215686274509803, 0.403921568627451, 0.403921568627451],
  [0.38823529411764707, 0.4, 0.4],
  [0.3803921568627451, 0.39215686274509803, 0.3843137254901961],
  [0.3803921568627451, 0.38823529411764707, 0.38823529411764707],
  ...
  [0.11764705882352941, 0.12941176470588237, 0.12549019607843137],
  [0.11764705882352941, 0.12941176470588237, 0.12549019607843137],
  [0.11764705882352941, 0.12941176470588237, 0.12549019607843137]]]

A quick and dirty fix is to use np.around():

img_list = np.around(img_np, 4).tolist()

The result is below the payload size limit.

mherzog
  • 1,085
  • 1
  • 12
  • 24