3

I am sending images from a Raspberry Pi to Cloud IoT Core. This is working, but I now realised that I also need the filename of the file (or the data that I have added to the filename), when storing it in Google Cloud Storage, but as far as I can see it is only the content of the file that is sent. I have tried to accomplish this with the code extract below, but this doesn't seem to work, which I guess is understandable seeing that it is a byte array inside a JSON object..?

Is there a way at all? Is the filename somehow retrievable when sending via Cloud Function to Storage?

with open(filename, 'rb') as f:
    imagestring = f.read()
byteArray = bytes(imagestring)

now = datetime.datetime.now()
datetime_formatted = now.strftime("%Y-%m-%d_%H:%M:%S")
filename = 'img_GM4_' + datetime_formatted +'.jpg'

payload = '{{ "ts": {}, filename: {}, "image": {}}}'.format(int(time.time()), filename, byteArray)

client.publish(_MQTT_TOPIC, payload, qos=1)
# Previously: client.publish(_MQTT_TOPIC, byteArray, qos=1)
Ingrid
  • 516
  • 6
  • 18

1 Answers1

0

In order to save the image in Cloud Storage you'll need to use a Cloud Function with a Cloud Pub/Sub trigger.

The Cloud Function should consist of the following steps:

  1. Decode the data from the data property of the PubSubMessage in a similar fashion to this function.
  2. Extract the value from the filename field and convert the value string from the image field to an actual image.
  3. Upload the image to Cloud Storage using one of the available methods. Like for example the Google Cloud Client Libraries for Python or the REST APIs and any of the available request libraries (e.g requests). You can found coding examples here on the documentation.

Finally, notice that according to best practices for Cloud Storage you should avoid timestamp-based for your filenames (this will constrain your throughput). In the context of Cloud Storage it is recommended to create filenames in the following manner. So take this last message into consideration if you notice performance issues within your application.

Daniel Ocando
  • 3,554
  • 2
  • 11
  • 19
  • This answer presupposes that the data is base64 encoded. I didn't do that since it wasn't necessary using MQTT. Do you mean it IS necessary to base64 encode, to be able to retrieve the file name? – Ingrid Feb 04 '20 at 09:57
  • Notice that in general the Pub/Sub message's data is stored as a base64-encoded string in the [data property](https://cloud.google.com/pubsub/docs/reference/rest/v1/PubsubMessage#FIELDS.data) of the PubsubMessage object. And that's why I included that step. If that is not the case you can omit the decode part and get the desired field. – Daniel Ocando Feb 04 '20 at 10:21
  • How do I access the filename field? I didn't know there was one in the `event` object of the `helloPubSub` function. (I'm writing my functions in Typescript btw). – Ingrid Feb 04 '20 at 13:46
  • The filename field should be on the data (which you need to parse) of the Cloud Pub/Sub topic (which you should be sending with `client.publish(_MQTT_TOPIC, payload, qos=1)`). If you are not receiving the data on Cloud Pub/Sub please refer to [this](https://cloud.google.com/iot/docs/support/troubleshooting#im_not_receiving_telemetry_data_on_cloud_pubsub) for troubleshooting. – Daniel Ocando Feb 04 '20 at 17:00
  • Ok. I get the data to Pub/Sub without problem, but I'd be really grateful for a description of how to (parse the data and) retrieve the filename. – Ingrid Feb 06 '20 at 09:31
  • Notice that the data you send is your `payload` variable and it is a string. Therefore you'll just need to deconstruct the way you initially created the `payload` variable. You could use a [regex](https://docs.python.org/3/library/re.html) to filter the value from the string or simply use the [string's split method](https://docs.python.org/3.7/library/stdtypes.html#str.split), as for example the following command `print(payload.split()[4][:-1])` should print the required filename field from your `payload` variable. – Daniel Ocando Feb 06 '20 at 22:13