GAE AttributeError: 'Credentials' object has no attribute 'with_subject'

Question

I have a python app I want to deploy on App Engine (2nd Generation Python 3.7) on which I use a Service Account with Domain-wide delegation enabled to access user data.

Locally I do:

import google.auth
from apiclient.discovery import build

creds, project = google.auth.default(
    scopes=['https://www.googleapis.com/auth/admin.directory.user', ],
)
creds = creds.with_subject(GSUITE_ADMIN_USER)

service = build('admin', 'directory_v1', credentials=creds)

This works good and, as far as I know it is the current way to do this when using Application Default Credentials (locally I have GOOGLE_APPLICATION_CREDENTIALS defined).

Problem is on GAE, when deployed, the call to with_subject raises: AttributeError: 'Credentials' object has no attribute 'with_subject'

I have enabled Domain-wide delegation on the GAE service account already.

What is different between the GOOGLE_APPLICATION_CREDENTIALS I use locally and the ones in GAE when both are service accounts with domain-wide delegation?

Where is .with_subject() on GAE?

The creds object received is of type compute_engine.credentials.Credentials.

Full traceback:

Traceback (most recent call last):
  File "/env/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/env/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 104, in init_process
    super(ThreadWorker, self).init_process()
  File "/env/lib/python3.7/site-packages/gunicorn/workers/base.py", line 129, in init_process
    self.load_wsgi()
  File "/env/lib/python3.7/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/env/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/env/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
    return self.load_wsgiapp()
  File "/env/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/env/lib/python3.7/site-packages/gunicorn/util.py", line 350, in import_app
    __import__(module)
  File "/srv/main.py", line 1, in <module>
    from config.wsgi import application
  File "/srv/config/wsgi.py", line 38, in <module>
    call_command('gsuite_sync_users')
  File "/env/lib/python3.7/site-packages/django/core/management/__init__.py", line 148, in call_command
    return command.execute(*args, **defaults)
  File "/env/lib/python3.7/site-packages/django/core/management/base.py", line 353, in execute
    output = self.handle(*args, **options)
  File "/srv/metanube_i4/users/management/commands/gsuite_sync_users.py", line 14, in handle
    gsuite_sync_users()
  File "/env/lib/python3.7/site-packages/celery/local.py", line 191, in __call__
    return self._get_current_object()(*a, **kw)
  File "/env/lib/python3.7/site-packages/celery/app/task.py", line 375, in __call__
    return self.run(*args, **kwargs)
  File "/srv/metanube_i4/users/tasks.py", line 22, in gsuite_sync_users
    creds = creds.with_subject(settings.GSUITE_ADMIN_USER)
AttributeError: 'Credentials' object has no attribute 'with_subject'"

Packages (partial list):

google-api-core==1.5.0
google-api-python-client==1.7.4
google-auth==1.5.1
google-auth-httplib2==0.0.3
google-cloud-bigquery==1.6.0
google-cloud-core==0.28.1
google-cloud-logging==1.8.0
google-cloud-storage==1.13.0
google-resumable-media==0.3.1
googleapis-common-protos==1.5.3
httplib2==0.11.3
oauthlib==2.1.0

Added the traceback as requested; Note that my code is basically the call to gsuite_sync_users() which is called later. (Celery is disabled now) — marc.fargas, Nov 08 '18 at 10:48

score 8 · Accepted Answer · answered Jul 18 '19 at 10:33

It is true that you cannot use the with_subject method with GAE or GCE credentials. However, there is a workaround that I was able to get working on my GCE server and I would assume this works with GAE default service accounts as well. The solution is to build new credentials using the service account identity with desired subject and scopes. A detailed guide can be found here, but I will also explain the process bellow.

Firstly, the service account needs permissions to create service account tokens for itself. This can be done by going to the projects IAM and admin > Service accounts page (make sure the info panel is visible, it can be toggled from the top right corner). Copy the service account email address and select the service account in question by ticking the checkbox. Now the info panel should have ADD MEMBER button. Click it and paste the service account email to the New members textbox. Click the Select role dropdown and choose the role Service Accounts -> Service Account Token Creator. You can check that the role is assigned with the following gcloud command:

gcloud iam service-accounts get-iam-policy [SERVICE_ACCOUNT_EMAIL]

Now to the actual Python code. This example is a slight modification from the documentation linked above.

from googleapiclient.discovery import build
from google.auth import default, iam
from google.auth.transport import requests
from google.oauth2 import service_account

TOKEN_URI = 'https://accounts.google.com/o/oauth2/token'
SCOPES = ['https://www.googleapis.com/auth/admin.directory.user']
GSUITE_ADMIN_USER = 'admin@example.com'

def delegated_credentials(credentials, subject, scopes):
    try:
        # If we are using service account credentials from json file
        # this will work
        updated_credentials = credentials.with_subject(subject).with_scopes(scopes)
    except AttributeError:
        # This exception is raised if we are using GCE default credentials

        request = requests.Request()

        # Refresh the default credentials. This ensures that the information
        # about this account, notably the email, is populated.
        credentials.refresh(request)

        # Create an IAM signer using the default credentials.
        signer = iam.Signer(
            request,
            credentials,
            credentials.service_account_email
        )

        # Create OAuth 2.0 Service Account credentials using the IAM-based
        # signer and the bootstrap_credential's service account email.
        updated_credentials = service_account.Credentials(
            signer,
            credentials.service_account_email,
            TOKEN_URI,
            scopes=scopes,
            subject=subject
        )
    except Exception:
        raise

    return updated_credentials


creds, project = default()
creds = delegated_credentials(creds, GSUITE_ADMIN_USER, SCOPES) 

service = build('admin', 'directory_v1', credentials=creds)

The try block will not fail if you have GOOGLE_APPLICATION_CREDENTIALS environment variable set with a path to a service account file. If the application is run on Google Cloud, there will be an AttributeError and it is handled by creating new credentials which have correct subject and scopes.

You can also pass None as the subject for delegated_credentials function and it creates the credentials without delegation so this function can be used with or without delegation.

I do not currently have an environment to test that. But the solution appears to solve my original problem so I'll mark it as answer. — marc.fargas, Jul 19 '19 at 11:52
It works for me, although, I noticed when I run this while deployed as a CloudFunction, it generates a good 20 lines of errors in the Logs before working properly. Would be nice if it didn't... — Michael, Aug 23 '19 at 18:11
This works for me on GAE. Thanks. Now I want to replicate it in nodejs. — JMKrimm, Jan 31 '22 at 12:38
i got the following error: :'Credentials' object has no attribute 'service_account_email.' FYI, I'm using identity federation credentials as: ```from google.auth import aws as google_auth_aws credentials = google_auth_aws.Credentials.from_info(json_config_info)``` — xanjay, May 02 '22 at 05:35
I had error got the following error: :'Credentials' object has no attribute 'service_account_email.' when I used my personal account. Check if you use a service account. — sacherus, Feb 28 '23 at 14:42

George · Answer 2 · 2018-11-16T16:49:31.983

0

@marc.fargas You may have a look at the googleapis/google-auth-library-python library on GitHub. You'll find some information relevant to the method in question:

The credentials are considered immutable. If you want to modify the scopes or the subject used for delegation, use :meth:with_scopes or :meth:with_subject:: scoped_credentials = credentials.with_scopes(['email']) delegated_credentials = credentials.with_subject(subject)

As you defined your Application Default Credentials with "GOOGLE_APPLICATION_CREDENTIALS", you were getting an instance of google.auth.service_account.Credentials which has the with_subject method.

While on App Engine, you are instead getting an instance of app_engine.Credentials, which does not have the with_subject method. This explains the observed behavior and the error you see.

According to the documentation on Domain-wide delegation, only Service account credentials can have domain-wide delegation.

edited Nov 16 '18 at 16:49

answered Nov 14 '18 at 23:24

George

1,488
1
10
13

The linked file is when using key files which should not be needed when using ADC (Application Default Credentials). Using key files it would work as you'd use another service account different of the GAE default one. – marc.fargas Nov 15 '18 at 08:15
1

.with_subject is what I'm trying to use. Thing is that the credentials passed down via ADC to the GAE environment do not have that hability for some reason. The issue is in the GAE credentials / ADC as that exact same code works outside of GAE when using $GOOGLE_APPLICATION_CREDENTIALS. I would really prefer to not have to use a key file as that would require that I push it somehow to GAE, either in the code (BAD) or using KMS (to much for too little?) – marc.fargas Nov 15 '18 at 08:17

score 0 · Answer 3 · answered Oct 09 '20 at 23:32

I was in a similar situation trying to run a query on a federated BQ table linked to a Google Sheet via a Cron job in GAE v2. The Google Drive scope which is required was not available to the default service account in GAE. vkopio's answer is great and I ended up using it as well because it looks cleaner, but here's another solution which doesn't require the Service Account Token Creator role to be assigned to the service account. I put it together while combing through the documentation for Cloud Functions (which uses underlying compute architecture similar to GAE) using the Rest API.

import requests

METADATA_URL = 'http://metadata.google.internal/computeMetadata/v1'
METADATA_HEADERS = {'Metadata-Flavor': 'Google'}
SERVICE_ACCOUNT = 'default'
SCOPES=['https://www.googleapis.com/auth/admin.directory.user']


def get_access_token(scopes):
    """
    Retrieves an access_token in App Engine for the default service account

    :param scopes: List of Google scopes as strings
    :return: access token as string
    """
    scopes_str = ','.join(scopes)
    url = f'{METADATA_URL}/instance/service-accounts/{SERVICE_ACCOUNT}/token?scopes={scopes_str}'
    # Request an access token from the metadata server.
    r = requests.get(url, headers=METADATA_HEADERS)
    r.raise_for_status()
    # Extract the access token from the response.
    access_token = r.json()['access_token']
    return access_token

I was able to use this access_token in my header for my request

headers = {'Authorization': f'Bearer {access_token}'}

r = requests.post(url, json=job_body, headers=headers)

where url points to the specific Rest endpoint I want to call with the appropriate configuration in job_body. Note that this does not work outside of the App Engine environment.

There was a way to create credentials using AccessTokenCredentials in oauth2client but it is now deprecated by Google, so this method requires using the Rest endpoints directly. Posting this answer so it's helpful for others who might not want to add any additional roles to the Service Account.

GAE AttributeError: 'Credentials' object has no attribute 'with_subject'

3 Answers3

Linked