5

I'm trying to access the custom metadata on a file in Google cloud storage from within a Cloud Function, but it always returns "None".

  • The file definitely has custom metadata on it - the custom metadata can be seen from the GCS browser.

  • I can access the custom metadata successfully if I run my code in a cloud shell environment.

  • But if I run the same code in a cloud function, then the returned blob.metadata is always "None".

What am I doing wrong? What do I have to do differently in a Cloud Function compared to a Cloud Shell.

from google.cloud import storage
client = storage.Client()
bucket = client.bucket(<my bucket name>)
blob = bucket.get_blob(<my filename>)
metadata = blob.metadata

Note also that blob.download_as_string() gets me the file contents correctly - it's just the custom metadata I can't get.

timhall
  • 51
  • 1
  • 3

5 Answers5

5

blob.metadata only returns Storage object's custom metadata (a dict). None means that there is no custom metadata. See the docs of metadata :

Retrieve arbitrary/application specific metadata for the object.

The documentation of Object resource (API) specify that metadata is :

User-provided metadata, in key/value pairs.

Note that custom metadata is different from fixed-key metadata, that you can also edit with Edit metadata button in Google Cloud Console. Fixed-key metadata contains :

  • Content-Type
  • Content-Encoding
  • Content-Disposition
  • Content-Language
  • Cache-Control

This particular kind of metadata can be accessed via blob.content_type, blob.content_encoding, ... (check a complete example).

To add custom metadata, just click Add item button on the same window (Edit metadata) or use gsutil (see Editing object metadata docs) :

gsutil setmeta -h "x-goog-meta-examplekey:examplevalue" gs://<your-bucket>
norbjd
  • 10,166
  • 4
  • 45
  • 80
  • 1
    I should have been clearer (my bad). 1) There is definitely custom metadata on the file - it has been added using exactly the gsutil command you suggest. 2) Viewing the file in the GCS browser shows that the custom metadata is indeed present. 3) Using the code that I included above in the context of a cloud shell returns that custom metadata 4) BUT as soon as I go into the context of a cloud function the metadata is always shown as None. I've edited above to be clearer. – timhall Jun 03 '19 at 11:03
2

In fact blob.metadata will not show the user metadata, you have to add blob.patch() and metadata will appear in blob.metadata variable

from google.cloud import storage
client = storage.Client()
bucket = client.bucket(<my bucket name>)
blob = bucket.get_blob(<my filename>)
blob.patch()
metadata = blob.metadata

To save user metadata, you can set blob.metadata and then call blob.patch to add or modify metadata, or blob.update to erase user metadata for this blob

McLooz
  • 21
  • 3
  • `blob.patch()` appears to be only to add metadata to upload. Personally, it does not add metadata for me to view. – GregK Sep 22 '20 at 17:13
0

Use blob.get_metadata, NOT blob.metadata

From the docs, blob.metadata does not make an HTTP request.

Sabito stands with Ukraine
  • 4,271
  • 8
  • 34
  • 56
Ted Romer
  • 11
  • 1
0

I ran into the same problem. But in my case, I have a trigger to run a cloud function whenever an object is created in a bucket. I needed to get the metadata as well using the code that you used, but it returns None as well.

I then changed the trigger to google.storage.object.metadataUpdate, so my Cloud Function would trigger whenever I set or update the metadata of an object. By doing that, I could get the metadata using the same code.

jdb
  • 1
0

blob.reload() can be used to refresh metadata from upstream.

blob = storage.Blob("name", bucket=bucket)
blob.reload()
metadata: dict = blob.metadata

Reload properties from Cloud Storage.

https://googleapis.dev/python/storage/2.2.1/blobs.html#google.cloud.storage.blob.Blob.reload

klden
  • 1