AWS Pass in variable into buildspec.yml from CodePipeline

Question

I have an AWS CodePipeline that invokes CodeBuild in the Build Stage.

The question is how do I pass in an environment variable from CodePipeline that can be read in the CodeBuild's buildspec.yml?

I know I can set environment variables in CodeBuild, but I want to use the same CodeBuild project for dev, qa, and prod environments. I don't see how I can pass an environment variable from CodePipeline that makes it all the way to the buildspec.yml

Example buildspec.yml

version: 0.1

phases:   
  build:
    commands:
      - npm install
      - npm build -- --env ${CURRENT_ENVIRONMENT}

Where CURRENT_ENVIRONMENT would be the variable I set in the CodePipeline Stage action.

For anyone interested, the way I got around this is you can set environment variables in CodeBuild that get passed to buildspec.yml. So as long as you can get your parameters into the AWS::CodeBuild::Project CloudFormation declaration, they are picked up when the buildspec.yml gets run. — user1432403, Feb 07 '17 at 20:20
This is a rather unfortunate limitation. I've added to an AWS forum thread about it [here](https://forums.aws.amazon.com/message.jspa?messageID=770850#770850) Consider chiming in to let AWS know if this is important to you. — jandersen, Mar 03 '17 at 20:26
how to use environment variables in buildspec.yml: npm build --env $THE_VARIABLE — ricka, Mar 15 '17 at 22:51

score 21 · Answer 1 · edited Jun 29 '20 at 14:22

21

As of today, you can set environment variables for CodeBuild build jobs in your pipeline. This improvement makes it possible to reuse the same build project for multiple actions and simplify deployments to staging and production environments in your pipeline.

name: Build
actions:
  - name: MyBuildJob
    actionTypeId:
      category: Build
      owner: AWS
      provider: CodeBuild
      version: '1'
    runOrder: 1
    configuration:
      ProjectName: my-build-project
      PrimarySource: MyApplicationSource1
      EnvironmentVariables: '[{"name":"CURRENT_ENVIRONMENT","value":"Production","type":"PLAINTEXT"},
                              {"name":"UseParamStore","value":"CURRENT_ENVIRONMENT","type":"PARAMETER_STORE"}]'

https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-CodeBuild.html#action-reference-CodeBuild-config

edited Jun 29 '20 at 14:22

Felipe Plets

7,230
3
36
60

answered Oct 16 '19 at 08:29

Vikyol

5,051
23
24

This is the best answer in 2021 in the "official AWS way". Note the (rather odd) JSON-as-string value of `EnvironmentVariables`. Also, all Cloudfront YAML keys should be CamelCase (I'm surprised the example works as is). – kino1 Jan 22 '21 at 16:53
To "pass in an environment variable from CodePipeline" as asked, use `{"name":"CURRENT_ENVIRONMENT","value":"#{StageNamespace.CurrentEnvironment}","type":"PLAINTEXT"}]` and add `Namespace: StageNamespace` to the action providing the variable. – kino1 Jan 22 '21 at 16:59
Hi, which file does this go in please? – Mar 25 '22 at 12:37

score 12 · Answer 2 · answered Oct 15 '18 at 10:31

You can actually pass environment variables in CodeBuild cloudformation as below:

Build:
    Type: AWS::CodeBuild::Project
    Properties:
      Name:
        !Sub Build-${AWS::StackName}
      Description: Build your project
      Environment:
        Type: LINUX_CONTAINER
        ComputeType: BUILD_GENERAL1_SMALL
        Image: node8
        EnvironmentVariables:
          - Name: CURRENT_ENVIRONMENT
            Type: PLAINTEXT
            Value: staging

And in your buildspec.yml you can reference the environment like this,

version: 0.2

phases:
  install:
    commands:
      - npm install

  build:
    commands:
      - npm build -- --env ${CURRENT_ENVIRONMENT}

Not helpful, as one would typically want to define a single `AWS::CodeBuild::Project` to use across all environments. Your approach would mean creating one `AWS::CodeBuild::Project` for each environment, where the only difference between them is the environment variables. — Jim Tough, Jan 12 '21 at 16:19

score 9 · Answer 3 · answered Jan 18 '17 at 18:52

9

This feature isn't available today.

A workaround would be to create a different CodeBuild project for each stage with different environment variables.

You can find details on using an environment variable in your builspec.yml commands here: http://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html

answered Jan 18 '17 at 18:52

TimB

1,457
8
10

15

One whole year, this feature still unavailable? – pablochacin Jan 23 '18 at 11:30
Wasn't able to figure out better idea than creating new build per environment... – BartusZak Oct 25 '19 at 12:50
this answer is invalid. There are three ways to get Env Vars into a CodeBuild step via a CloudFormation template now: in the CodeBuild resource `Environment.EnvironmentVariables` section, in the Action step of a CodePipeline resource that uses CodeBuild, or in the CodeBuilds buildSpec file itself: https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html#build-spec-ref-syntax – airtonix May 22 '20 at 03:43

score 8 · Answer 4 · answered Mar 11 '21 at 14:25

8

Create a Codebuild project.
Create a pipeline project inside Codepipline.
Add a CodeBuild action and define your variables in this action.
For example:
Variable name is: TAG
Variable value is: staging

Using the variable you defined in the Codepipline Buildspec file in the Codebuild project:

the logging results:

answered Mar 11 '21 at 14:25

Eliran Azulay

201
4
3

Tried this and worked like a charm. This one should be the accepted answer as it explains what the question asks, in the best and simplest way with today's (August 2021) available options in AWS. – SebaGra Aug 12 '21 at 16:09

score 5 · Answer 5 · answered Mar 01 '19 at 18:18

You can use build environment variables to conditionally pass from buildspec to npm build by detecting which CodeBuild job or CodePipeline job is running. If you have one CodePipeline listening to /dev commits and another listening to /master commits, for example, this works perfectly.

Here's an example which runs a different PROD vs DEV build:

  build:
    commands:
   - |
     if expr "${CODEBUILD_BUILD_ARN}" : ".*build/MyProjectDev-" >/dev/null; then
       yarn run build-dev;
     fi
   - |
     if expr "${CODEBUILD_BUILD_ARN}" : ".*build/MyProject-" >/dev/null; then
       yarn run build-prod;
     fi

score 4 · Answer 6 · answered Nov 08 '18 at 22:29

If you don't want to use CF, you can set ENV vars on your CodeBuild project in the AWS UI.

In AWS, go to your code builder project, in the top right corner click "edit" and select "environment." On the edit environment page, click the "Additional Configuration" drop down. In there you will see inputs for "name" and "value." "Name" is where you set your ENV and "value" is where you set your value for that variable.

Example: set API_ENV in "name" and development in "value." Then, in your buildspec.yml, you can use $API_ENV.

score 4 · Answer 7 · answered Jul 11 '20 at 23:58

I am sure there are many answers posted above but putting some extra info which I found useful.

CodePipeline started supporting the environment variable at the Pipeline level where some of have native support via Pipeline (CodePipeline-generated pipeline variables)

So for example if we need to get the CommitId in any stage then we can declare the environment variable as per below.

Give any name to variable namespace on source stage for example "SourceVariables"
Use that namespace in any stage as "#{SourceVariables.CommitId}" in this case its getting use in build stage.

Also we can generate our own environment variable from any stage which will be available to use in next stages. More info is available here.

Going back to now original question, we can set the environment variable on build action which will help us to use it along with build spec file. But still it will not allow us to dynamically pass it during every push of code from source. And due to that it require two steps like update pipeline and then trigger the Pipeline so that it can go to respective stage.

This is indeed correct, but you didn't emphasize enough that the **Environment Variables** must be set from `CodePipeline -> Edit: Build -> Environment variables - optional`. If you set these variables in `CodeBuild -> Edit -> Environment -> Additional configuration -> Environment variables` it WON'T WORK! This is so dumb, but I spent too much time on this myself. — Raphael Setin, Sep 14 '22 at 19:38

score 1 · Answer 8 · edited Jun 29 '20 at 14:24

I created a small python script to parse the $CODEBUILD_INITIATOR variable that is passed. Below is the buildspec.yml and below that is the python script that I include with the build and call.

      build:
        commands:
          - |
            PIPELINE_ENV=$(python3 codebuild_env_parser.py $CODEBUILD_INITIATOR)
            OUTPUT_STATUS=$?
            if [ "$OUTPUT_STATUS" = "0" ]; then
              echo "Success finding a valid environment from codebuild_env_parser.py."
            else
              echo "Failure finding a valid environment from codebuild_env_parser.py. Check the script to see if the codepipeline env was passed correctly."
            fi

Python script (codebuild_env_parser.py):

    import sys
    
    
    def main():
        args = sys.argv
        if len(args) == 2:
            env_list = ["dev", "prod"]
            pipeline_invoker = args[1].lower()
            code_pipeline_name = pipeline_invoker.split("codepipeline/")[1]
            env_name = code_pipeline_name.split("-project-name")[0]
            if env_name in env_list:
                print("{}".format(env_name))
                sys.exit(0)
            else:
                sys.exit(1)
        else:
            sys.exit(1)
    
    
    main()

You'll have to tweak some variable values here if you want this to work. Namely, "-project-name".

score 0 · Answer 9 · edited Apr 01 '19 at 22:31

I've created a lambda function that updates an existing codebuild project's environment variables. You can the start the build (codebuild.start) after updating the variables. Looks something like this (nodejs):

var params = {
    "name": "Build-project-name",
    "description": "Build this project.",

    "environment": {
        "type": "LINUX_CONTAINER",
        "image": "aws/codebuild/standard:1.0",
        "computeType": "BUILD_GENERAL1_LARGE",
        "environmentVariables": [
        {
            "name": "MY_ENVIRONMENT_VARIABLE",
            "value": "VALUE_OF_ENVIRONMENT_VARIABLE",
            "type": "PLAINTEXT"
        }]
    }
}

codebuild.updateProject(params, function(err, data) {
    if (err) {
        console.log(err, err.stack); // an error occurred
    }
    else     {
        console.log(data);           // successful response
    }
});

score 0 · Answer 10 · answered May 09 '19 at 07:59

The CodeBuild initiator is an environment variable in CodeBuild and this can be used to read in the CodePipeline name.

So if you include your environment in your CodePipeline name as a suffix (-dev or -prod, for instance), then you can parse it out like so.

version: 0.2
phases:
  build:
    commands:
      - CURRENT_ENVIRONMENT=`echo $CODEBUILD_INITIATOR | cut -d '-' -f2 | tr '[:upper:]' '[:lower:]'`
      - echo "My env is $CURRENT_ENVIRONMENT"

AWS Pass in variable into buildspec.yml from CodePipeline

10 Answers10

Linked