Vision API: How to get JSON-output

Question

I'm having trouble saving the output given by the Google Vision API. I'm using Python and testing with a demo image. I get the following error:

TypeError: [mid:...] + is not JSON serializable

Code that I executed:

import io
import os
import json
# Imports the Google Cloud client library
from google.cloud import vision
from google.cloud.vision import types

# Instantiates a client
vision_client = vision.ImageAnnotatorClient()


# The name of the image file to annotate
file_name = os.path.join(
    os.path.dirname(__file__),
    'demo-image.jpg') # Your image path from current directory

# Loads the image into memory
with io.open(file_name, 'rb') as image_file:
    content = image_file.read()
    image = types.Image(content=content)

# Performs label detection on the image file
response = vision_client.label_detection(image=image)
labels = response.label_annotations


print('Labels:')
for label in labels:
    print(label.description, label.score, label.mid)

with open('labels.json', 'w') as fp:
   json.dump(labels, fp)

the output appears on the screen, however I do not know exactly how I can save it. Anyone have any suggestions?

score 16 · Accepted Answer · edited Aug 01 '21 at 18:13

FYI to anyone seeing this in the future, google-cloud-vision 2.0.0 has switched to using proto-plus which uses different serialization/deserialization code. A possible error you can get if upgrading to 2.0.0 without changing the code is:

object has no attribute 'DESCRIPTOR'

Using google-cloud-vision 2.0.0, protobuf 3.13.0, here is an example of how to serialize and de-serialize (example includes json and protobuf)

import io, json
from google.cloud import vision_v1
from google.cloud.vision_v1 import AnnotateImageResponse

with io.open('000048.jpg', 'rb') as image_file:
    content = image_file.read()

image = vision_v1.Image(content=content)
client = vision_v1.ImageAnnotatorClient()
response = client.document_text_detection(image=image)

# serialize / deserialize proto (binary)
serialized_proto_plus = AnnotateImageResponse.serialize(response)
response = AnnotateImageResponse.deserialize(serialized_proto_plus)
print(response.full_text_annotation.text)

# serialize / deserialize json
response_json = AnnotateImageResponse.to_json(response)
response = json.loads(response_json)
print(response['fullTextAnnotation']['text'])

Note 1: proto-plus doesn't support converting to snake_case names, which is supported in protobuf with preserving_proto_field_name=True. So currently there is no way around the field names being converted from response['full_text_annotation'] to response['fullTextAnnotation'] There is an ~~open~~ closed feature request for this: googleapis/proto-plus-python#109

Note 2: The google vision api doesn't return an x coordinate if x=0. If x doesn't exist, the protobuf will default x=0. In python vision 1.0.0 using MessageToJson(), these x values weren't included in the json, but now with python vision 2.0.0 and .To_Json() these values are included as x:0

This is the actual solution. Anyone having trouble with converting google api response to json should simply look at this. — Maaz Bin Musa, Mar 16 '21 at 20:09
It's a shame that the google-cloud-vision 2.0.0 have no easy way of exporting to a json. Probably fine to use the vision_v1 code, but I'm edgy. — yangliu2, Mar 27 '21 at 14:07

score 4 · Answer 2 · answered Dec 12 '18 at 13:24

Maybe you were already able to find a solution to your issue (if that is the case, I invite you to share it as an answer to your own post too), but in any case, let me share some notes that may be useful for other users with a similar issue:

As you can check using the the type() function in Python, response is an object of google.cloud.vision_v1.types.AnnotateImageResponse type, while labels[i] is an object of google.cloud.vision_v1.types.EntityAnnotation type. None of them seem to have any out-of-the-box implementation to transform them to JSON, as you are trying to do, so I believe the easiest way to transform each of the EntityAnnotation in labels would be to turn them into Python dictionaries, then group them all into an array, and transform this into a JSON.

To do so, I have added some simple lines of code to your snippet:

[...]

label_dicts = [] # Array that will contain all the EntityAnnotation dictionaries

print('Labels:')
for label in labels:
    # Write each label (EntityAnnotation) into a dictionary
    dict = {'description': label.description, 'score': label.score, 'mid': label.mid}

    # Populate the array
    label_dicts.append(dict) 

with open('labels.json', 'w') as fp:
   json.dump(label_dicts, fp)

Thank you to remind me :) I do have found an Answer and now posted. — Bruna B, Dec 19 '18 at 13:25

score 3 · Answer 3 · answered Apr 03 '19 at 05:13

3

There is a library released by Google

from google.protobuf.json_format import MessageToJson

webdetect = vision_client.web_detection(blob_source) jsonObj = MessageToJson(webdetect)

answered Apr 03 '19 at 05:13

Robert Robinson

407
5
14

1

Any idea how to parse it back to Message ? – Jimmar Dec 17 '19 at 15:35
Ok I will check that – Robert Robinson Dec 24 '19 at 04:43

score 1 · Answer 4 · answered Dec 19 '18 at 13:25

I was able to save the output with the following function:

# Save output as JSON
def store_json(json_input):
    with open(json_file_name, 'a') as f:
        f.write(json_input + '\n')

And as @dsesto mentioned, I had to define a dictionary. In this dictionary I have defined what types of information I would like to save in my output.

with open(photo_file, 'rb') as image:
    image_content = base64.b64encode(image.read())
    service_request = service.images().annotate(
        body={
            'requests': [{
                'image': {
                    'content': image_content
                },
                'features': [{
                    'type': 'LABEL_DETECTION',
                    'maxResults': 20,
                },
                    {
                        'type': 'TEXT_DETECTION',
                        'maxResults': 20,
                    },
                        {
                            'type': 'WEB_DETECTION',
                            'maxResults': 20,
                        }]
            }]
        })

score 0 · Answer 5 · answered Dec 20 '18 at 13:03

The objects in the current Vision library lack serialization functions (although this is a good idea).

It is worth noting that they are about to release a substantially different library for Vision (it is on master of vision's repo now, although not released to PyPI yet) where this will be possible. Note that it is a backwards-incompatible upgrade, so there will be some (hopefully not too much) conversion effort.

That library returns plain protobuf objects, which can be serialized to JSON using:

from google.protobuf.json_format import MessageToJson
serialized = MessageToJson(original)

You can also use something like protobuf3-to-dict

Vision API: How to get JSON-output

5 Answers5