Google Cloud Functions - How to securely store service account private key when using Google Source Repository?

Question

I use Google Source Repository to store my Google Cloud Functions. (Git repo hosted by Google, basically)

One of my function needs to access a private Google Sheet file, I therefore created a Service Account. (With way too many rights since it's so hard to understand what exact rights we should give to a service account, and so hard to update later on, but I digress)

Now, it's clearly not recommended to store the Service Account JSON file in the git repository itself for obvious reasons. Here is what it looks like (stripped from values)

{
  "type": "service_account",
  "project_id": "",
  "private_key_id": "",
  "private_key": "",
  "client_email": "",
  "client_id": "",
  "auth_uri": "",
  "token_uri": "",
  "auth_provider_x509_cert_url": "",
  "client_x509_cert_url": ""
}

I have been looking at environment variables to configure for a Functions or something alike but didn't find anything. Tracking the key (and therefore potentially duplicating that file on several repositories) really doesn't sound such a good idea. But I haven't found any "proper" way to do it yet. And due to the way Google Functions work, I can't think of anything else but env variables.

score 9 · Accepted Answer · answered Aug 17 '19 at 05:58

9

My solution when using cloud function with a service account is:

Encrypt your service account credential json file using Cloud KMS/vault and upload it to Cloud Storage.
Fetch service account credential json file from Cloud Storage and decrypt it using a Cloud KMS service account which has encrypt/decrypt permission.
Parse service account credential json file at runtime and get private_key, client_email and projectId.
Pass these three secret variables to the client library

We store config variables as environment variables for cloud function, they are plain text, but it's ok. Because they are not secret things.

We must not store secret things like plain text, e.g cloud function environment variables.

answered Aug 17 '19 at 05:58

Lin Du

88,126
95
281
483

This seems to be the most comprehensive and reliable answer so far, I'm therefore marking it as "Answer". I haven't tested it, and won't, since I switched to AWS a long time ago. Future readers: If this answer doesn't answer the question then please let me know (ping) in the comments and I shall review it. :) – Vadorequest Aug 23 '19 at 12:55
1

This is only needed if you need to create a client with a specific service account JSON file. eg FCM Client. if your Google function uses APIs that get the credentials from the current context then this answer is for you. https://stackoverflow.com/questions/55671256/how-to-use-gcp-service-account-in-google-cloud-function – ozOli Sep 30 '19 at 15:59

score 7 · Answer 2 · answered Feb 03 '18 at 22:14

7

You can upload the service account file along with your functions and use it from within your code. It will remain secure there. Most developers will use a .gitignore or equivalent mechanism to keep that file from being added to source control. There is an example of loading service account credentials from Firebase samples. (If you're not using the Firebase SDK, you'll have to be mindful to convert the function definition to the Cloud style.

You could also use an env var, but you'll have to take special care in quoting and escaping the values to make sure they get to your function correctly. It's kind of a hassle, but doable.

answered Feb 03 '18 at 22:14

Doug Stevenson

297,357
32
422
441

So, if I understand correctly, your solution is to somehow deploy the `service-account-credentials.json` file in the Google Function without adding it to the Google Source Repository, by relying on `firebase/gcloud` CLI? (I don't use Firebase, indeed. How would you use an env variable? Can you add an example or doc? Thanks. – Vadorequest Feb 03 '18 at 23:52
1

Yes, that's what I'm saying. The docs for the Firebase CLI: https://firebase.google.com/docs/cli/ – Doug Stevenson Feb 04 '18 at 01:09
hey @DougStevenson! I am actually a bit confused about the Firebase SDK reference in here. Are you essentially suggesting to just deploy the service account key as part of the cloud function to GCP? – jayBana Jan 07 '20 at 14:45
This is exactly what I did with my project that initialized the app based on service account credentials from my numerous environments. I added a .gitignore entry to my service account JSON files folder. I deploy the cloud functions directly. – Saurabh Aug 30 '21 at 16:21

score 2 · Answer 3 · answered Aug 26 '20 at 08:41

2

As of January 2020, Google has released a Secret Manager, which is described as:

Secret Manager is a new Google Cloud service that provides a secure and convenient method for storing API keys, passwords, certificates, and other sensitive data. Secret Manager provides a central place and single source of truth to manage, access, and audit secrets across Google Cloud.

For Cloud Functions, there is a tutorial here on how to create a secret and then retrieve it from a cloud function.

answered Aug 26 '20 at 08:41

neal

343
3
10

1

Notice that even when using Secret Manager, one should question "But where do I store credentials to access Secret Manager?" or "How determine who can access to the secrets?". All examples you linked rely on Google's ADC mechanics to access secrets, that uses the service account of the cloud function to handle the initial authentication. Your answer is a good replacement for @slideshowp2 solution for generic credentials (out of GCP scope), but to access GCP resources, I think people should rely solely in ADC. – Diego Queiroz Aug 26 '20 at 12:16

score 0 · Answer 4 · answered Feb 07 '18 at 18:44

0

Here you could find how to provide credentials to your application, using the environment variable GOOGLE_APPLICATION_CREDENTIALS.

answered Feb 07 '18 at 18:44

Katayoon

592
5
13

1

This is a solution, but won't work with Cloud Functions since you don't control the hosting system. You can use this in your local machine, or a server you own, but not with cloud functions. Also, I don't like it, because if you need to use several different credentials that aren't related (like several services each providing something requiring their own credential) then you cannot use it. (unless you make a big credential which contain all those credentials but... micro services, right?) – Vadorequest Feb 10 '18 at 11:09
Using the environment variable GOOGLE_APPLICATION_CREDENTIALS is at your preference. You could also save your credential file on a bucket (providing the appropriate access level) and address it. Then you can obtain and provide service account credentials manually in your code as mentioned in this documentation: https://cloud.google.com/docs/authentication/production#obtaining_and_providing_service_account_credentials_manually – Katayoon Feb 12 '18 at 20:05
2

Everybody knows about the link to google cloud docs but that only works if you run the app locally. The problem is that the docs do not explain how to set the json file in production. – ttfreeman Apr 21 '19 at 22:43
How about setting the Cloud Function's `GOOGLE_APPLICATION_CREDENTIALS` environment variable using a local credentials.json file while deploying the function? – pdoherty926 Nov 13 '19 at 15:23

score 0 · Answer 5 · answered Apr 28 '19 at 15:58

This is how I solved this problem. First create a logic in a file keys.js to determine whether you are in development or production (and create corresponding ./dev.js and ./prod.js files, where you should include ./dev.js in .ignore file to make sure it's not uploaded to your github remote):

if (process.env.NODE_ENV === "production") {
  module.exports = require("./prod");
} else {
  module.exports = require("./dev");
}

Second, you require your keys.js file where the logic above resides and create a credential object based on the data received from keys.js:

const credentials = {
  type: keys.googleType,
  project_id: keys.googleProjectId,
  private_key_id: keys.googlePrivateKeyId,
  private_key: keys.googlePrivateKey,
  client_email: keys.googleClientEmail,
  client_id: keys.googleClientId,
  auth_uri: keys.googleAuthUri,
  token_uri: keys.googleTokenUri,
  auth_provider_x509_cert_url: keys.googleAuthProviderX509CertUrl,
  client_x509_cert_url: keys.googleClientX509CertUrl
};

Now, for every google cloud service you can use the following example patterns:

  const storage = new Storage({
    project_id: credentials.project_id,
    credentials
  });
  const client = new textToSpeech.TextToSpeechClient({
    project_id: credentials.project_id,
    credentials
  });
...
etc.

score 0 · Answer 6 · answered Aug 26 '20 at 05:12

I think you don't really need to store any keyfile within your code.

Your function runs with a designated service account (usually the "App Engine default service account", but you can change in Advanced Settings). If you have specific needs, you should create a service account specific for your function and grant to it all the permission it needs.

In your function, the authentication will occur automatically using Application Default Credentials, so you don't have to care to anything (forget about environment variables, keyfiles, or anything). Just assure you're using the Google Cloud Client Libraries for your language that they'll handle everything for you implicitly.

Particularly, I avoid creative solutions as the one @slideshowp2 proposed. But I agree they have its uses (eg. suppose I need to store credentials for an external system, out of GCP scope. In this scenario, his solution may be a way to go), but to consume only Google services, let's keep it simple.

Google Cloud Functions - How to securely store service account private key when using Google Source Repository?

6 Answers6

Linked

Related