7

Please could someone help me with a query related to permissions on the Google cloud platform? I realise that this is only loosely programming related so I apologise if this is the wrong forum!

I have a project ("ProjectA") written in Python that uses Googles cloud storage and compute engine. The project has various buckets that are accessed using python code from both compute instances and from my home computer. This project uses a service account which is a Project "owner", I believe it has all APIs enabled and the project works really well. The service account name is "master@projectA.iam.gserviceaccount.com".

Recently I started a new project that needs similar resources (storage, compute) etc, but I want to keep it separate. The new project is called "ProjectB" and I set up a new master service account called master@projectB.iam.gserviceaccount.com. My code in ProjectB generates an error related to access permissions and is demonstrated even if I strip the code down to these few lines:

The code from ProjectA looked like this:

from google.cloud import storage

client = storage.Client(project='projectA')
mybucket = storage.bucket.Bucket(client=client, name='projectA-bucket-name')

currentblob = mybucket.get_blob('somefile.txt')

The code from ProjectB looks like this:

from google.cloud import storage

client = storage.Client(project='projectB')
mybucket = storage.bucket.Bucket(client=client, name='projectB-bucket-name')

currentblob = mybucket.get_blob('somefile.txt')

Both buckets definitely exist, and obviously if "somefile.text" does not exist then currentblob is None, which is fine, but when I execute this code I receive the following error:

Traceback (most recent call last):
  File .... .py", line 6, in <module>
    currentblob = mybucket.get_blob('somefile.txt')
  File "C:\Python27\lib\site-packages\google\cloud\storage\bucket.py", line 599, in get_blob
    _target_object=blob,
  File "C:\Python27\lib\site-packages\google\cloud\_http.py", line 319, in api_request
    raise exceptions.from_http_response(response)
google.api_core.exceptions.Forbidden: 403 GET https://www.googleapis.com/storage/v1/b/<ProjectB-bucket>/o/somefile.txt: master@ProjectA.iam.gserviceaccount.com does not have storage.objects.get access to projectB/somefile.txt.

Notice how the error message says "ProjectA" service account doesn't have ProjectB access - well, I would somewhat expect that but I was expecting to use the service account on ProjectB!

Upon reading the documentation and links such as this and this, but even after removing and reinstating the service account or giving it limited scopes it hasnt helped. I have tried a few things:

1) Make sure that my new service account was "activated" on my local machine (where the code is being run for now):

gcloud auth activate-service-account master@projectB.iam.gserviceaccount.com --key-file="C:\my-path-to-file\123456789.json"

This appears to be successful.

2) Verify the list of credentialled accounts:

gcloud auth list

This lists two accounts, one is my email address (that I use for gmail, etc), and the other is master@projectB.iam.gserviceaccount.com, so it appears that my account is "registered" properly.

3) Set the service account as the active account:

gcloud config set account master@projectB.iam.gserviceaccount.com

When I look at the auth list again, there is an asterisk "*" next to the service account, so presumably this is good.

4) Check that the project is set to ProjectB:

gcloud config set project projectB

This also appears to be ok.

Its strange that when I run the python code, it is "using" the service account from my old project even though I have changed seemingly everything to refer to project B - Ive activated the account, selected it, etc.

Please could someone point me in the direction of something that I might have missed? I don't recall going through this much pain when setting up my original project and Im finding it so incredibly frustrating that something I thought would be simple is proving so difficult.

Thank you to anyone who can offer me any assistance.

Paul
  • 528
  • 5
  • 17
  • Granting the role Owner to service account credentials is not good practice. Use least privilege for service accounts. – John Hanley Jul 16 '19 at 06:30

3 Answers3

7

I'm not entirely sure, but this answer is from a similar question on here: Permission to Google Cloud Storage via service account in Python

Specifying the account explicitly by pointing to the credentials in your code. As documented here:

Example from the documentation page:

def explicit():
    from google.cloud import storage

    # Explicitly use service account credentials by specifying the private key
    # file.
    storage_client = storage.Client.from_service_account_json(
        'service_account.json')

    # Make an authenticated API request
    buckets = list(storage_client.list_buckets())
    print(buckets)
Spiderbro
  • 359
  • 2
  • 10
  • 1
    This answer provides the recommended method. Where possible do not use environment variables (GOOGLE_APPLICATION_CREDENTIALS) as this is environment-specific and can be modified by external applications/users. – John Hanley Jul 16 '19 at 06:27
  • Thank you Spiderbro (and everyone else) for the prompt responses and comments. I am keen to try this solution as soon as I am home from work a bit later today. Just one clarification though and possibly a silly question, but where it says 'service_account.json') I should replace this string with the absolute path and filename to my .json keyfile? Otherwise I dont see how it is uniquely identifying my json credentials file? – Paul Jul 16 '19 at 12:42
  • Yes. Since you're pointing to a specific file, you should give the correct path. If the file is in the current working directory, you can use the above code ( if the filename is the same, of course). Otherwise, provide the absolute path. – Spiderbro Jul 16 '19 at 18:09
  • 1
    Thank you for this. My code is working on the new Project now - thank you so much for your help! – Paul Jul 16 '19 at 18:54
0

Don't you have a configured GOOGLE_APPLICATION_CREDENTIALS env variable which points project A's SA?

jszule
  • 414
  • 2
  • 4
0

The default behavior of Google SDK is to takes the service account from the environment variable GOOGLE_APPLICATION_CREDENTIALS.

If you want to change the account you can do something like:

from google.cloud import storage
credentials_json_file = os.environ.get('env_var_with_path_to_account_json')
client= storage.Client.from_service_account_json(credentials)

The above assumes you have creates a json account file like in: https://cloud.google.com/iam/docs/creating-managing-service-account-keys and that the json account file is in the environment variable env_var_with_path_to_account_json

This way you can have 2 account files and decide which one to use.

Erez Ben Harush
  • 833
  • 9
  • 26