Secret environment variables in Cloudbuild (with out files), how?

Question

I am creating a CI/CD pipeline in Cloud Build of a very basic Node.js app with deployment to GCP appengine standard.

None-secret environment variables are stored in app.yaml file. But of course I don't want to put my secrets there. In fact I don't want to put them in any file any where (encrypted or not) since this file will end up on the AppEngine instance and can be "viewed" by a "bad admin". There are many samples out there that suggests to encrypt/decrypt complete files (and some times even code) but I don't want to go down that path.

I am looking for a way to set secret environment variables "in memory" as part of the CI/CD pipeline. Anyone?

I added none secrets in the app.yaml file (env_variables) - works fine Added encrypted secrets into my cloudbuild.yaml file (secrets) - no error Added secretEnv: into a build steps but value don't end up as process.env.[KEY] in app engine

cloudbuild.yaml

steps:
- name: 'gcr.io/cloud-builders/npm'
  args: ['install']
  dir: "appengine/hello-world/standard"
 - name: "gcr.io/cloud-builders/gcloud"
  args: ["app", "deploy", "test-app.yaml"]
  dir: "appengine/hello-world/standard"
  secretEnv: ['API_KEY', 'API_URL']

secrets:
- kmsKeyName: projects/XXXXXXXX/locations/global/keyRings/customintegrations-secrets/cryptoKeys/integration-secrets
  secretEnv:
    API_KEY: XXQAoHgKKoHBKOURrUU2RqU+ki8XyqmTjz+ns+MEWp5Kx3hQBpgSQgATFQ5yRdW4m1TLNqNRIdHIqVJi8tn8jFrtlHIEouOzNDe/ASlOT0ZQBfl9Rf7xlvOHAa667poBq2hEoMNvOclxUQ==
    API_URL: YYQAoHgKKklo08ZsQF+/8M2bmi9nhWEtb6klyY4rNthUhSIhQ8oSQQATFQ5ywKOxaM/TLwGDmvMtCpl/1stXOOK0kgy42yipYbw/J/QZL68bMat1u4H3Hvp/GMbUVIKEb9jwUtN2xvbL

I was hoping that the secretEnv: ['API_KEY', 'API_URL'] would make the decrypted values accessable in code (process.env.API_KEY) in app engine.

I have been wanting to do the same but the only way I found was to encrypt a file in Cloud Storage with a KMS the Cloud Build Service Account has access to and download+decrypt. This in order to centralize (to some extent) the secrets. — polve, Jun 12 '19 at 19:35
Thanks. How did you get the values in the file into env variables so you could access them in code like, process.env.API_KEY? — cjt, Jun 12 '19 at 20:04
This [tutorial](https://cloud.google.com/cloud-build/docs/securing-builds/use-encrypted-secrets-credentials#encrypting_a_file_using_the_cryptokey) explains the process, so once the file is decrypted during the build step you should be able to reference it as normal. — Corinne White, Jun 13 '19 at 14:41
@CorinneWhite the document you refer to doesn’t put the secretEnv: ['API_KEY', 'API_URL'] decrypted values as ENV variables. I have already tried this. Are you telling me that there is absolutely no way I can achieve what I need. I.e. secrets in ENV variables with NO file containing the secrets in clear-text deployed to GAE.(read above for the details) — cjt, Jun 14 '19 at 12:04
Using the command 'gcloud builds submit' you can substitute parameters in the build specification: https://cloud.google.com/sdk/gcloud/reference/builds/submit#--substitutions. Have a look at the documentation, and let me know if it's a valid option for you. — Maxim, Jun 18 '19 at 10:53
Does this answer your question? [How to set environment variables using Google Cloud Build or other method in Google App Engine Standard Environment?](https://stackoverflow.com/questions/52840187/how-to-set-environment-variables-using-google-cloud-build-or-other-method-in-goo) — user3396492, Nov 21 '20 at 13:51

score 3 · Answer 1 · answered Jun 29 '19 at 06:26

Here is a full tutorial on how to securely store env vars in your cloud build (triggers) settings and import them into your app.

Basically there are three steps:

Add your env vars to the 'variables' section in one of your build trigger settings

Screenshot of where to add variables in build triggers

By convention variables set in the build trigger must begin with an underscore (_)
Configure cloudbuild.yaml (on the second step in the code example) to read in variables from your build trigger, set them as env vars, and write all env vars in a local .env file

Add couldbuild.yaml (below) to your project root directory

steps:
- name: node:10.15.1
  entrypoint: npm
  args: ["install"]
- name: node:10.15.1
  entrypoint: npm
  args: ["run", "create-env"]
  env:
    - 'MY_SECRET_KEY=${_MY_SECRET_KEY}'
- name: "gcr.io/cloud-builders/gcloud"
  args: ["app", "deploy"]
timeout: "1600s"

Add create-env script to package.json

"scripts": {
  "create-env": "printenv > .env"
},

Read env vars from .env to your app (config.js)

Install dotenv package

npm i dotenv -S

Add a config.js to your app

// Import all env vars from .env file
require('dotenv').config()

export const MY_SECRET_KEY = process.env.MY_SECRET_KEY

console.log(MY_SECRET_KEY) // => Hello

Done! Now you may deploy your app by triggering the cloud build and your app will have access to the env vars.

Whilst you're getting the variables into process.env if I'm not mistaken this solution would still create a new .env file, which a "bad admin" would still be able to get hold of and get the secrets as referenced in the initial post with the google storage option. — TommyBs, Sep 18 '19 at 12:03
@TommyBs I guess the goal here is to store the secrets somewhere on GCP account and not in the code locally. So the secrets are only going to be as secure as your GCP login. However, I personally feel setting up google storage to store secrets and having to fetch them in your app is a lot more work. — highfivebrian, Sep 23 '19 at 23:55
@highfivebrian Is this still valid? I'm not able to create .env file from cloudbuild.yaml. It was working couple months ago for sure. Same code I'm trying to deploy and it is not creating .env file in app engine. Any thoughts? — Kalpesh Patel, Dec 06 '19 at 19:12

score 2 · Answer 2 · answered Jul 25 '22 at 10:16

Using secrets from Secrets Manager

Your sample would become:

steps:
- name: 'gcr.io/cloud-builders/npm'
  args: ['install']
  dir: "appengine/hello-world/standard"
- name: "gcr.io/cloud-builders/gcloud"
  args: ["app", "deploy", "test-app.yaml"]
  dir: "appengine/hello-world/standard"
  secretEnv: ['API_KEY', 'API_URL']

availableSecrets:
  secretManager:
  - versionName: projects/$PROJECT_ID/secrets/<secret name>/versions/latest
    env: API_KEY 
  - versionName: projects/$PROJECT_ID/secrets/<secret name 2>/versions/latest
    env: API_URL

score 0 · Answer 3 · answered Oct 13 '21 at 12:02

Add a cloud trigger step in your cloudbuild.yaml to add place holders in your app.yaml file

    steps:
    - name: "gcr.io/cloud-builders/gcloud"
      secretEnv: ['API_KEY','API_URL']
      entrypoint: 'bash'   args: 
      - -c
      - |
        echo $'\n  API_KEY: '$$API_KEY >> app.yaml
        echo $'\n  API_URL: '$$API_URL >> app.yaml
        gcloud app deploy
    availableSecrets:   secretManager:
      - versionName: projects/012345678901/secrets/API_KEY
        env: 'API_KEY'
      - versionName: projects/012345678901/secrets/API_URL
        env: 'API_URL'

look following reference app.yaml

runtime: nodejs
service: serviceone
env_variables:
  PROJECT_ID: demo
  PORT: 8080

Reference by: https://stackoverflow.com/users/13763858/cadet

**look following reference app.yaml** `runtime: nodejs main: cmd service: serviceone env_variables: PROJECT_ID: demo PORT: 8080` — Aadesh kale, Oct 13 '21 at 12:05

Secret environment variables in Cloudbuild (with out files), how?

3 Answers3