AWS Elastic Beanstalk and Secret Manager

Question

Does anyone know is it possible to pass a secret value as an environment variable in elastic beanstalk? The alternative obviously is to use the sdk in our codebase but I want to explore the environment variable approach first

Cheers Damien

Answer 1

Per @Ali's answer, it is not built-in at this point. However, it is relatively easy to use .ebextensions and the AWS cli. Here is an example that extracts a secret to a file, according to an MY_ENV environment variable. This value could then be set to an environment variable, but keep in mind environment variables are specific to the shell. You'd need to pass them in to anything you are launching.

  10-extract-htpasswd:
    env:
      MY_ENV: 
        "Fn::GetOptionSetting":
          Namespace: "aws:elasticbeanstalk:application:environment"
          OptionName: MY_ENV
    command: |
      aws secretsmanager get-secret-value --secret-id myproj/$MY_ENV/htpasswd --region=us-east-1 --query=SecretString --output text > /etc/nginx/.htpasswd
      chmod o-rwx /etc/nginx/.htpasswd
      chgrp nginx /etc/nginx/.htpasswd

This also requires giving the EB service role IAM permissions to the secrets. i.e. A policy like:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "xxxxxxxxxx",
            "Effect": "Allow",
            "Action": "secretsmanager:GetSecretValue",
            "Resource": "arn:aws:secretsmanager:us-east-1:xxxxxxxxxxxx:secret:myproj*"
        }
    ]
}

Answer 2

As above answers mention, there is still no built-in solution if you want to do this in Elastic Beanstalk. However a work around solution is to use "platform hook". Unfortunately it is poorly documented at this point.

To store your secret, best solution is to create a custom secret in AWS-Secret-Manager. In secret manager you can create a new secret by clicking "Store a new secret", then selecting "Other type of secret" and entering your secret key/value (see ). At the next step you need to provide a Secret Name (say "your_secret_name") and you can leave everything else to their default settings.

Then, you need to allow Elastic Beanstalk to get this secret. You can do it by creating a new IAM policy, for instance with this content:

{
"Version": "2012-10-17",
"Statement": [
    {
        "Sid": "Getsecretvalue",
        "Effect": "Allow",
        "Action": [
            "secretsmanager:GetResourcePolicy",
            "secretsmanager:GetSecretValue",
            "secretsmanager:DescribeSecret",
            "secretsmanager:ListSecretVersionIds"
        ],
        "Resource": "your-secret-arn"
    }
]}

You need to replace "your-secret-arn" with your secret ARN which you can get on AWS-secret-manager interface. Then, you need to add the policy you created to EB roles (it should be either "aws-elasticbeanstalk-ec2-role" or "aws-elasticbeanstalk-service-role").

Finally you need to add a hook file in your application. From the root of your application, add file ".platform/hooks/prebuild/your_hook.sh". Content of your file can be something like this:

#!/bin/sh
export your_secret_key=$(aws secretsmanager get-secret-value --secret-id your-secret-name --region us-east-1 | jq -r '.SecretString' | jq -r '.your_secret_key')

touch .env
{
  printf "SECRET_KEY=%s\n" "$your_secret_key"
  # printf whatever other variable you want to pass
} >> .env

Obviously you need to replace "your_secret_name" and the other variable by your own values and set the region to the region where your secret is stored (if it is not us-east-1). And don't forget to make it executable ("chmod +x your_hook.sh").

This assumes that your application can load its env from a .env file (which works fine with docker / docker-compose for example).

Another option is to store the variable in an ".ebextensions" config file but unfortunately it doesn't seem to work with the new Amazon Linux 2 platform. What's more you should not store sensitive information such as credentials directly in your application build. Builds of the application can be accessed by anyone with Elastic Beanstalk Read Access and they are also store unencrypted on S3.

With the hook approach, the secret is only stored locally on your Elastic Beanstalk underlying EC2 instances, and you can (should!) restrict direct SSH access to them.

Answer 3

Unfortunately, EB doesn't support secrets at this point, this might be added down the road. You can use them in your environment variables as the documentation suggests but they will appear in plain text in the console. Another, and IMO better, approach would be to use ebextensions, and use AWS CLI commands to grab secrets from the secrets manager, which needs some set up (e.g. having AWS CLI installed and having your secrets stored in SM). You can set these as environment variables in the same eb configuration. Hope this helps!

Answer 4

I'm just adding to @kaliatech's answer because while very helpful, it had a few gaps that left me unable to get this working for a few days. Basically you need to add a config file to the .ebextensions directory of your EB app, which uses a container_commands section retrieve your secret (in JSON format) and output it as a .env. file into the /var/app/current directory of the EC2 instances where your app's code lives:

# .ebextensions/setup-env.config
container_commands:
  01-extract-env:
    env:
      AWS_SECRET_ID:
        "Fn::GetOptionSetting":
          Namespace: "aws:elasticbeanstalk:application:environment"
          OptionName: AWS_SECRET_ID
      AWS_REGION: {"Ref" : "AWS::Region"}
      ENVFILE: .env

    command: >
        aws secretsmanager get-secret-value --secret-id $AWS_SECRET_ID --region $AWS_REGION |
        jq -r '.SecretString' |
        jq -r 'to_entries|map("\(.key)=\(.value|tostring)")|.[]' > $ENVFILE

Note: this assumes the AWS_SECRET_ID is configured in the app environment, but it can easily be hardcoded here as well.

All the utils needed for this script to work are already baked into the EC2 Linux image, but you'll need to grant permissions to the IamInstanceProfile role (usually named aws-elasticbeanstalk-ec2-role) which is assumed by EC2 to allow it access SecretManager:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "SecretManagerAccess",
            "Effect": "Allow",
            "Action": "secretsmanager:GetSecretValue",
            "Resource": "arn:aws:secretsmanager:ap-southeast-2:xxxxxxxxxxxx:secret:my-secret-name*"
        }
    ]
}

Finally, to debug any issues encountered during EC2 instance bootstrap, download the EB logs and check the EC2 log files at /var/log/cfn-init.log and /var/log/cfn-init-cmd.log.

Answer 5

This answer only applies if you're using code pipeline.

I think you can add a secret in the environment variables section now

Answer 6

If you use AWS CodeBuild use pre_build, add the following commands in your project's buildspec.yml to retrieve your environment variables from AWS Secrets Manager use sed to do some substituting/formatting and append them to .ebextensions/options.config's aws:elasticbeanstalk:application:environment namespace:

phases:
  pre_build:
    commands:
      - secret=$(aws secretsmanager get-secret-value --secret-id foo-123 --region=bar-xyz --query=SecretString --output text)
      - regex=$(cat ./sed_substitute)
      - echo $secret | sed "${regex}" >> .ebextensions/options.config

Bit of a hack but the sed_substitute used in the commands above used to get the correct indentation/formatting that .ebextensions/options.config demands was:

s/",/\n /g; s/":/": /g; s/{"/ /g; s/"}//g; s/"//g;