12

This is a bit of a newbie question, but I've just gotten started with GCP provisioning using Terraform / Terragrunt, and I find the workflow with obtaining GCP credentials quite confusing. I've come from using AWS exclusively, where obtaining credentials, and configuring them in the AWS CLI was quite straightforward.

Basically, the Google Cloud Provider documentation states that you should define a provider block like so:

provider "google" {
  credentials = "${file("account.json")}"
  project     = "my-project-id"
  region      = "us-central1"
  zone        = "us-central1-c"
}

This credentials field shows I (apparently) must generate a service account, and keep a JSON somewhere on my filesystem.

However, if I run the command gcloud auth application-default login, this generates a token located at ~/.config/gcloud/application_default_credentials.json; alternatively I can also use gcloud auth login <my-username>. From there I can access the Google API (which is what Terraform is doing under the hood as well) from the command line using a gcloud command.

So why does the Terraform provider require a JSON file of a service account? Why can't it just use the credentials that the gcloud CLI tool is already using?

By the way, if I configure Terraform to point to the application_default_credentials.json file, I get the following errors:

Initializing modules...

Initializing the backend...

Error: Failed to get existing workspaces: querying Cloud Storage failed: Get https://www.googleapis.com/storage/v1/b/terraform-state-bucket/o?alt=json&delimiter=%2F&pageToken=&prefix=projects%2Fsomeproject%2F&prettyPrint=false&projection=full&versions=false: private key should be a PEM or plain PKCS1 or PKCS8; parse error: asn1: syntax error: sequence truncated

Scott Crooks
  • 1,523
  • 4
  • 24
  • 39

3 Answers3

12

if I configure Terraform to point to the application_default_credentials.json file, I get the following errors:

The credentials field in provider config expects a path to service account key file, not user account credentials file. If you want to authenticate with your user account try omitting credentials and then running gcloud auth application-default login; if Terraform doesn't find your credentials file you can set the GOOGLE_APPLICATION_CREDENTIALS environment variabe to point to ~/.config/gcloud/application_default_credentials.json.

Read here for more on the topic of service accounts vs user accounts. For what it's worth, Terraform docs explicitly advice against using application-default login:

This approach isn't recommended- some APIs are not compatible with credentials obtained through gcloud

Similarly GCP docs state the following:

Important: For almost all cases, whether you are developing locally or in a production application, you should use service accounts, rather than user accounts or API keys.

Aleksi
  • 4,483
  • 33
  • 45
  • 1
    Ah, I see now, thank you very much! I was so confused about why this happens. The part that you mention from the Terraform Docs ("This approach isn't recommended- some APIs are not compatible with credentials obtained through gcloud") still worries me a bit. That implies that *some* actions I cannot do through a regular `gcloud` command. That seems very strange. – Scott Crooks Aug 12 '19 at 07:36
  • 1
    actually it looks like a user *must* use a service account. When I do your recommended steps, and point `GOOGLE_APPLICATION_CREDENTIALS` to `~/.config/gcloud/application_default_credentials.json`, it still gives me the error above, saying that it's not in PEM format. – Scott Crooks Aug 12 '19 at 10:56
  • Oh right, my bad. Does `application-default login` still work if you unset the `GOOGLE_APPLICATION_CREDENTIALS` variable? – Aleksi Aug 12 '19 at 11:02
  • FWIW, the linked 'GCP Docs' no longer contain the 'you should use service accounts' phrase. User accounts don't seem to be discouraged, and appear to be intended for developers. – Matt Drees Feb 17 '20 at 17:12
  • 5
    Currently, during development, using the `gcloud auth application-default login` method is recommended both by [Google](https://cloud.google.com/iam/docs/best-practices-service-accounts#development) (Instead, let them use your credentials by running ... `gcloud auth application-default login`) and by [Terraform](https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/getting_started#configuring-the-provider) (The easiest way to do this is to run `gcloud auth application-default login`) – Jacek J Aug 09 '22 at 10:06
0

Change the credentials to point directly to the file location. Everything else looks good.

Example: credentials = "/home/scott/gcp/FILE_NAME"

Scott Pike
  • 1
-2

Still it is not recommended to use gcloud auth application-default login, Best best approaches are

https://www.terraform.io/docs/providers/google/guides/provider_reference.html#credentials-1

krish
  • 1,081
  • 9
  • 10
  • 1
    this is wrong, the same link, https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference#credentials-1 states: If you are using Terraform on your workstation we recommend that you install gcloud and authenticate using User Application Default Credentials ("ADCs") as a primary authentication method. You can enable ADCs by running the command gcloud auth application-default login. – cryanbhu Nov 29 '21 at 02:44