Blob.generate_signed_url() failing to AttributeError

Question

So I'm trying to produce temporary globally readable URLs for my Google Cloud Storage objects using the google-cloud-storage Python library (https://googlecloudplatform.github.io/google-cloud-python/latest/storage/blobs.html) - more specifically the Blob.generate_signed_url() method. I doing this from within a Compute Engine instance in a command line Python script. And I keep getting the following error:

AttributeError: you need a private key to sign credentials.the credentials you are currently using <class 'oauth2cl
ient.service_account.ServiceAccountCredentials'> just contains a token. see https://google-cloud-python.readthedocs
.io/en/latest/core/auth.html?highlight=authentication#setting-up-a-service-account for more details.

I am aware that there are issues with doing this from within GCE (https://github.com/GoogleCloudPlatform/google-auth-library-python/issues/50) but I have created a new Service Account credentials following the instructions here: https://cloud.google.com/storage/docs/access-control/create-signed-urls-program and my key.json file most certainly includes a private key. Still I am seeing that error.

This is my code:

keyfile = "/path/to/my/key.json"
credentials = ServiceAccountCredentials.from_json_keyfile_name(keyfile)
expiration = timedelta(3) # valid for 3 days
url = blob.generate_signed_url(expiration, method="GET",
                               credentials=credentials)

I've read through the issue tracker here https://github.com/GoogleCloudPlatform/google-cloud-python/issues?page=2&q=is%3Aissue+is%3Aopen and nothing related jumps out so I am assuming this should work. Cannot see what's going wrong here.

I am currently hunting the solution for this, as well, because I need a way to sign upload URLs without physical access to a key file. I got this working when a service account key (.json) is configured to `GOOGLE_APPLICATION_CREDENTIALS`, but, perhaps like you, I want my code to operate with the default credentials implicit to instances on GCP. Did you solve it? — ThatsAMorais, Jan 31 '18 at 00:29
When connecting local machine to storage, we can use credentials from json. But when reading file on app engine from storage- i dont know how to get credentials. docs say default service account should just work fine. But I always get error ' need private key to sign' — Aseem, Aug 11 '19 at 18:07

score 18 · Answer 1 · answered Dec 12 '18 at 18:37

18

I was having the same issue. Ended up fixing it by starting the storage client directly from the service account json.

storage_client = storage.Client.from_service_account_json('path_to_service_account_key.json')

I know I'm late to the party but hopefully this helps!

answered Dec 12 '18 at 18:37

tomasn4a

575
4
15

3

You saved my life! For anyone else reading this in the future, I found this only applied to creation of v2 signed URLs, when using v4 this did not apply – Jack Casey Jun 21 '19 at 18:04
Doesn't work; that's what I did to get the error in the first place. – Zimano Nov 17 '20 at 14:44
1

doesn't match with workload identity enabled – SegFault Dec 02 '20 at 10:57
1

What if I dont want to save sv account key in GIT. is there a way ? – Aseem Jul 15 '21 at 06:42
worked in v4 for me – mr blu Jan 30 '22 at 13:19
This worked in v4 for me too. Also if you already have the service account as a dictionary loaded, you can use `from_service_account_info` – Seperman Feb 16 '22 at 20:06

score 12 · Answer 2 · edited May 14 '23 at 14:23

12

Currently, it's not possible to use blob.generate_signed_url without explicitly referencing credentials. (Source: Class Blob - generate_signed_url) However, you can do a workaround, as seen here, which consists of:

signing_credentials = compute_engine.IDTokenCredentials(
    auth_request,
    "",
    service_account_email=credentials.service_account_email
)
signed_url = signed_blob_path.generate_signed_url(
    expires_at_ms,
    credentials=signing_credentials,
    version="v4"
)

edited May 14 '23 at 14:23

t-mart

890
11
27

answered Sep 16 '19 at 20:38

Mito

121
1
5

Thanks! I had to put the `service_account_email` in the environment, as `credentials.service_account_email` was coming back as _default_ (I think) – Steven Jun 18 '20 at 11:13
This code works tho afterwards I had to follow this to get the right permissions in google cloud https://stackoverflow.com/a/62327597/494560 – jcroll Nov 21 '20 at 21:49
How do I get this "auth_request" variable? – Alberto Vitoriano Oct 20 '22 at 21:35

score 3 · Answer 3 · answered Nov 22 '22 at 01:35

A much complete snippet for those asking where other elements come from. cc @AlbertoVitoriano

    from google.auth.transport import requests
    from google.auth import default, compute_engine
    
    credentials, _ = default()
    
    # then within your abstraction
    auth_request = requests.Request()
    credentials.refresh(auth_request)
    
    signing_credentials = compute_engine.IDTokenCredentials(
        auth_request,
        "",
        service_account_email=credentials.service_account_email
    )
    signed_url = signed_blob_path.generate_signed_url(
        expires_at_ms,
        credentials=signing_credentials,
        version="v4"
    )

Blob.generate_signed_url() failing to AttributeError

3 Answers3

Linked