0

I'm encountering a 'NotFound' error while attempting to use the Blob.rewrite() method to copy files from one Google Cloud Storage bucket to another within a Google Cloud Function. The error message I'm receiving is as follows:

NotFound: 404 POST https://storage.googleapis.com/storage/v1/b/ca-app-corp-finance-dev-444/o/process%2Ffin_essbase%2Fdata%2Fincoming%2Fessbase%2Fkey.ppk/rewriteTo/b/gcf-v2-uploads-632430838198-us-east1/o/key.ppk?prettyPrint=false: No such object: ca-app-corp-finance-dev-444/process/fin_essbase/data/incoming/essbase/key.ppk

Here's the detailed description of my setup and the problem:

I've set up a Google Cloud Function that triggers whenever a new file is uploaded to a source bucket (gcf-v2-uploads-632430838198-us-east1). The intention is to copy the uploaded file to a destination bucket (ca-app-corp-finance-dev-444) under the specific path process/fin_essbase/data/incoming/essbase/. To achieve this, I'm using the Blob.rewrite() method within the function. Below is an excerpt from my Python code within the Cloud Function:

from google.cloud import storage

def transfer_file(data, context):
    source_bucket_name = "gcf-v2-uploads-632430838198-us-east1"
    destination_bucket_name = "ca-app-corp-finance-dev-444"
    target_path = "process/fin_essbase/data/incoming/essbase/"
    filename = "key.ppk"

    storage_client = storage.Client()
    source_bucket = storage_client.bucket(source_bucket_name)
    destination_bucket = storage_client.bucket(destination_bucket_name)

    source_blob = source_bucket.blob(filename)
    destination_blob = destination_bucket.blob(target_path + filename)

    # Check if the destination blob already exists
    if destination_blob.exists():
        print(f"Destination blob {destination_blob.name} already exists.")
    else:
        try:
            # Perform a rewrite operation to copy the content
            token = source_blob.rewrite(destination_blob)

            # Wait for the rewrite to complete
            token.result()

            # Delete the source blob
            source_blob.delete()

            print(f"File {filename} transferred successfully.")
        except Exception as e:
            print(f"An error occurred: {e}")

I've verified the existence of the source object in the source bucket and ensured that the bucket names and file paths are accurate. The service account associated with the Cloud Function has the necessary permissions for reading from the source bucket and writing to the destination bucket.

Despite this, I'm encountering the 'NotFound' error. Could there be any potential misconfigurations or additional factors causing this issue? I'm hoping to receive insights or guidance on how to troubleshoot and resolve this problem.

Someone
  • 178
  • 2
  • 12

1 Answers1

1

Few possible reasons why you might see a NotFound error when trying to copy files between buckets using the Blob.rewrite() method in a Google Cloud Function:

  1. The file you want to copy might not be in the destination bucket yet. The Blob.rewrite() method won't create it unless it's missing. If you're sure the file should be there, try making it yourself before using Blob.rewrite().

  2. The service account that the Cloud Function uses might not be allowed to add files to the destination bucket. To check this, go to the Cloud Console, find the service account linked to your Cloud Function, and see if it has permissions to view and create objects in the destination bucket.

  3. There could be a problem with how the Cloud Function connects to the destination bucket. You can look into this by running a command in Cloud Shell:

    gcloud compute instances describe INSTANCE_NAME | grep network

    This helps you see if the Cloud Function is on the same network as the bucket. If not, it might not be able to reach the bucket. gcloud compute instances describe

If you've made sure the destination file exists, the service account has the right permissions, and there are no network problem, there's a chance the Blob.rewrite() method might have a bug. If that's the case, you can let Google Cloud Support know by reporting the problem.

Here are some extra links that could be useful:

Cloud Functions documentation

Storage API documentation

Cloud Functions troubleshooting guide

Julia
  • 512
  • 6