Upload ndarray(image in OpenCV) on to google cloud storage as a .jpg or .png

Question

I have a similar issues like How to upload a bytes image on Google Cloud Storage from a Python script.

I tried this

from google.cloud import storage
import cv2
from tempfile import TemporaryFile
import google.auth
credentials, project = google.auth.default()
client = storage.Client()
# https://console.cloud.google.com/storage/browser/[bucket-id]/
bucket = client.get_bucket('document')
# Then do other things...
image=cv2.imread('/Users/santhoshdc/Documents/Realtest/15.jpg')
with TemporaryFile() as gcs_image:
    image.tofile(gcs_image)
    blob = bucket.get_blob(gcs_image)
    print(blob.download_as_string())
    blob.upload_from_string('New contents!')
    blob2 = bucket.blob('document/operations/15.png')

    blob2.upload_from_filename(filename='gcs_image')

This is the error that's posing up

> Traceback (most recent call last):   File
> "/Users/santhoshdc/Documents/ImageShapeSize/imageGcloudStorageUpload.py",
> line 13, in <module>
>     blob = bucket.get_blob(gcs_image)   File "/Users/santhoshdc/.virtualenvs/test/lib/python3.6/site-packages/google/cloud/storage/bucket.py",
> line 388, in get_blob
>     **kwargs)   File "/Users/santhoshdc/.virtualenvs/test/lib/python3.6/site-packages/google/cloud/storage/blob.py",
> line 151, in __init__
>     name = _bytes_to_unicode(name)   File "/Users/santhoshdc/.virtualenvs/test/lib/python3.6/site-packages/google/cloud/_helpers.py",
> line 377, in _bytes_to_unicode
>     raise ValueError('%r could not be converted to unicode' % (value,)) ValueError: <_io.BufferedRandom name=7> could not be
> converted to unicode

Can anyone guide me what's going wrong or what I'm doing incorrectly?

@A.Queue Please look into this – Santhosh Apr 05 '18 at 09:56 — Santhosh, Apr 05 '18 at 09:56

Santhosh · Answer 1 · 2018-04-09T05:21:31.077

As suggested by @A.Queue in(gets deleted after 29 days)

from google.cloud import storage
import cv2
from tempfile import TemporaryFile

client = storage.Client()

bucket = client.get_bucket('test-bucket')
image=cv2.imread('example.jpg')
with TemporaryFile() as gcs_image:
    image.tofile(gcs_image)
    gcs_image.seek(0)
    blob = bucket.blob('example.jpg')
    blob.upload_from_file(gcs_image)

The file got uploaded,but uploading a numpy ndarray doesn't get saved as an image file on the google-cloud-storage

PS:

numpy array has to be convert into any image format before saving.

This is fairly simple, use the tempfile created to store the image, here's the code.

with NamedTemporaryFile() as temp:

    #Extract name to the temp file
    iName = "".join([str(temp.name),".jpg"])

    #Save image to temp file
    cv2.imwrite(iName,duplicate_image)

    #Storing the image temp file inside the bucket
    blob = bucket.blob('ImageTest/Example1.jpg')
    blob.upload_from_filename(iName,content_type='image/jpeg')

    #Get the public_url of the saved image 
    url = blob.public_url

was the issue solved in the end? I am asking because in the other question you have wrote that it was a issue with the path, but here you say it is unreadable. — A.Queue, Apr 07 '18 at 15:49
@A.Queue Yes the issue was solved. `numpy array` direct upload will not be stored in image format hence unreadable. I have updated the code for `numpy array` first converted and stored in `tempfile` then uploaded — Santhosh, Apr 09 '18 at 05:19
but you need to delete the file after you uplaoded. Otherwise the disk can be overfilled — Temak, Aug 24 '20 at 16:52

Alfe · Answer 2 · 2018-04-05T09:35:37.497

1

You are calling blob = bucket.get_blob(gcs_image) which makes no sense. get_blob() is supposed to get a string argument, namely the name of the blob you want to get. A name. But you pass a file object.

I propose this code:

with TemporaryFile() as gcs_image:
    image.tofile(gcs_image)
    gcs_image.seek(0)
    blob = bucket.blob('documentation-screenshots/operations/15.png')
    blob.upload_from_file(gcs_image)

edited Apr 05 '18 at 09:35

answered Apr 04 '18 at 13:44

Alfe

56,346
20
107
159

`Traceback (most recent call last): File "/Users/santhoshdc/Documents/ImageShapeSize/imageGcloudStorageUpload.py", line 34, in blob2.upload_from_filename(filename='gcs_image') File "/Users/santhoshdc/.virtualenvs/test/lib/python3.6/site-packages/google/cloud/storage/blob.py", line 1021, in upload_from_filename with open(filename, 'rb') as file_obj: FileNotFoundError: [Errno 2] No such file or directory: 'gcs_image'`. This is the error I'm getting on removing those three lines – Prashant Panwar Apr 04 '18 at 14:10
Urgs. `filename='gcs_image'` makes no sense. I'll add code I think might work to my answer. Hang on. – Alfe Apr 04 '18 at 15:07
@PrashantPanwar I added example code which should do the trick. – Alfe Apr 04 '18 at 15:15
The program seems to work fine no errors, but there isn't any file being uploaded into the bucket. Trying to upload an image through `gsutil` in the terminal seems to upload the image. Have no idea where it is going wrong? – Prashant Panwar Apr 05 '18 at 05:27
Maybe you need to `close()` the blob? I didn't look that up. – Alfe Apr 05 '18 at 09:01
Getting AttributeError: ‘Blob’ object has no attribute ‘close’. – Prashant Panwar Apr 05 '18 at 09:22
Maybe slashes are not allowed in the blob names? As I said, I'm not familiar with the Google Cloud Services. – Alfe Apr 05 '18 at 09:30
Scanning through https://googlecloudplatform.github.io/google-cloud-python/latest/storage/blobs.html I found examples which inspired this: `from google.cloud.storage import Blob` — `blob = Blob('15.png', bucket)` — `blob.upload_from_file(gcs_image)`. Could you test this? – Alfe Apr 05 '18 at 09:35
Tried with [this code](https://cloud.google.com/storage/docs/uploading-objects#storage-upload-object-python) as well but still same issue. Getting confused here – Prashant Panwar Apr 05 '18 at 09:39
1

Try running this code https://pastebin.com/Jr6k3SW2. It's mostly the same but without `google.auth` which is unnecessary here as google.cloud is handling everything in the background. It will be a good point of reference as I have tried it and it works. – A.Queue Apr 05 '18 at 11:25
Where are you executing the script? Have you set all the necessary environment variables and checked that the bucket exists? I know this questions are to obvious but it never hurts to check. – A.Queue Apr 05 '18 at 11:29
I'm explaining my work in another post. You might get a clear picture there. @A.Queue – Santhosh Apr 05 '18 at 11:50
[link](https://stackoverflow.com/questions/49671827/upload-an-image-from-cloud-function-python-to-google-cloud-storage) @A.Queue – Santhosh Apr 05 '18 at 11:58
[pastebin.com/Jr6k3SW2](https://pastebin.com/Jr6k3SW2) Worked as expected Thank You @A.Queue but please check the above link. – Santhosh Apr 05 '18 at 12:29
Tried to upload a PIL `JpegImageFile`. I am getting `AttributeError: 'JpegImageFile' object has no attribute 'read'` @A.Queue – Santhosh Apr 06 '18 at 06:38
@A.Queue Could you please look into the link [here](https://github.com/python-pillow/Pillow/issues/3079#event-1560370122) – Santhosh Apr 06 '18 at 10:48
[`.tofile`](https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.tofile.html) in `image.tofile(gcs_image)` is saving an ndarray into your blob. So `blob2 = bucket.blob('document/operations/15.png')` isn't really a png file. If you want to write it as an image instead of ndarray maybe you can use [`imwrite`](https://docs.opencv.org/3.0-beta/modules/imgcodecs/doc/reading_and_writing_images.html?highlight=imread#imwrite). – A.Queue Apr 07 '18 at 15:59

Upload ndarray(image in OpenCV) on to google cloud storage as a .jpg or .png

2 Answers2

Linked