Dynamic References to Specify Secret Manager Values in AWS Cloudformation

Question

Is there anyway we can pass dynamic references to Secret Manager to AWS Launch Config User Data?

Here is the code snippet I tried:

"SampleLaunchConfig": {
            "Type": "AWS::AutoScaling::LaunchConfiguration",
             "Properties": {
                "ImageId": {
                    "Fn::FindInMap": [
                        "AWSRegionArch2AMI",
                        {
                            "Ref": "AWS::Region"
                        },
                        "AMI"
                    ]
                },
                "UserData": {
                    "Fn::Base64": {
                        "Fn::Join": [
                            "",
                            [
                                "#!/bin/bash -xe\n",
                                "yum update -y\n",
                                "useradd -p <<pwd>>{{resolve:secretsmanager:Credentials:SecretString:userName}}\n",
                                "\n"
                            ]
                        ]
                    }
                }
        }
    }

Seems error in getting the useradd: invalid user name '{{resolve:secretsmanager:Credentials:SecretString:userName}}'

How can I pass Secret Manager secret value to cloudformation user data ?

direvus · Answer 1 · 2019-02-18T00:57:36.867

20

It seems that {{resolve:...}} dynamic references are only expanded in certain contexts within a template.

There is no precise information in the AWS docs about exactly where in a template you can use these references. The current wording with regard to {{resolve:secretsmanager:...}} says:

"The secretsmanager dynamic reference can be used in all resource properties"

However this is contradicted by your example, and I've also observed dynamic references failing to resolve inside of CloudFormation::Init data.

I have an active Support case open with AWS about this, they have agreed that the behaviour of dynamic references is inadequately documented. I'll update this answer as I learn more.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/dynamic-references.html#dynamic-references-secretsmanager

edited Feb 18 '19 at 00:57

answered Feb 13 '19 at 06:12

direvus

362
2
6

4

Hey, I wonder if AWS support managed to provide something meaningful back? – m1keil Feb 28 '19 at 03:54
14

AWS Support have confirmed that dynamic references do **not** resolve within `UserData` or `CloudFormation::Init` properties. They do resolve in many other resource properties, and where the property supports dynamic references, you can use them inside of function calls (!Sub, !Join, etc.). What is still unclear is exactly which properties support these references and which do not. My support case is still open. – direvus Mar 01 '19 at 00:36
Jammed into this page of documentation is a table detailing which properties will resolve: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/dynamic-references.html Look for the "Resources that support dynamic parameter patterns for secure strings" section. – chrismo Apr 13 '22 at 19:17
2

Hi @chrismo, that table is referring to `ssm-secure` dynamic references. The current question is about Secrets Manager which is not the same thing. – direvus Apr 13 '22 at 23:29

score 8 · Answer 2 · answered Dec 05 '18 at 18:26

I am not sure why this is not expanded correctly for you. However, you probably do not want CFN to expand your secret in the user data because the password would be embedded in the base64 encoded user data script which is visible in the EC2 console.

Instead you should take advantage of the fact that you have a script that executes on the host and call secrets manager at script execution time (warning untested):

"SampleLaunchConfig": {
        "Type": "AWS::AutoScaling::LaunchConfiguration",
         "Properties": {
            "ImageId": {
                "Fn::FindInMap": [
                    "AWSRegionArch2AMI",
                    {
                        "Ref": "AWS::Region"
                    },
                    "AMI"
                ]
            },
            "UserData": {
                "Fn::Base64": {
                    "Fn::Join": [
                        "",
                        [
                            "#!/bin/bash -xe\n",
                            "yum update -y\n",
                            "yum install -y jq\n",
                            !Sub "useradd -p `aws --region ${AWS::Region} secretsmanager get-secret-value --secret-id Credentials --query SecretString --output text | jq -r .passwordKey` `aws --region ${AWS::Region} secretsmanager get-secret-value --secret-id Credentials --query SecretString --output text | jq -r .userName`\n",
                            "\n"
                        ]
                    ]
                }
            }
    }
}

This is not ideal since it expands the password on the command line. It might be made more secure by putting the password in a file first and reading it from there and then shredding the file.

@PawełPrażak The value of `SecretString` is actually a JSON string. In other words, JSON nested in JSON. So, here the `--query SecretString` gets the value of `SecretString` from the response and then `jq` is getting the value of the `userName` key in the JSON string. — marcguyer, May 05 '20 at 14:45

score 3 · Answer 3 · answered Apr 10 '19 at 04:30

I can confirm that @JoeB's "warning untested" answer works, with the caveat that the machine in question must have permission to read the secret. You'll need something like

  MyInstancePolicy:
    Type: AWS::IAM::Policy
    Properties:
      PolicyName: MyPolicy
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          -
            Effect: Allow
            Action:
              - secretsmanager:GetSecretValue
            Resource: !Join
              - ''
              - - !Sub "arn:aws:secretsmanager:${AWS::Region}:"
                - !Sub "${AWS::AccountId}:secret:Credentials-??????"

Note a couple of things:

Unlike S3 buckets, you can't do arn:aws:secretsmanager:::secret.... If you don't want to explicitly declare the region and account, you need to use a wildcard. Buried at the bottom of Using Identity-based Policies (IAM Policies) for Secrets Manager

If you don't care about the region or account that owns a secret, you must specify a wildcard character * (not an empty field) for the region and account ID number fields of the ARN.

Perhaps less important, and less likely to cause unexpected failures, but still worth note:

Using '??????' as a wildcard to match the 6 random characters that are assigned by Secrets Manager avoids a problem that occurs if you use the '*' wildcard instead. If you use the syntax "another_secret_name-*", it matches not just the intended secret with the 6 random characters, but it also matches "another_secret_name-a1b2c3". Using the '??????' syntax enables you to securely grant permissions to a secret that doesn't yet exist.

score 1 · Answer 4 · answered Aug 22 '19 at 21:46

Variant on @JoeB's answer:

Resources:
  SampleLaunchConfig:
    Type: AWS::AutoScaling::LaunchConfiguration
    Properties:
      ImageId: !FindInMap [ AWSRegionArch2AMI, !Ref: 'AWS::Region', AMI ]
      UserData:
        Fn::Base64: !Sub |
          #!/bin/bash -xe
          exec > >(tee /var/log/user-data.log | logger -t user-data) 2>&1

          yum update -y
          yum install -y jq

          username=$(aws secretsmanager get-secret-value --secret-id Credentials \
                                                         --query SecretString \
                                                         --region ${AWS::Region} --output text | jq -r .userName)
          password=$(aws secretsmanager get-secret-value --secret-id Credentials \
                                                         --query SecretString \
                                                         --region ${AWS::Region} --output text | jq -r .passwordKey)
          useradd -p "$password" $username

UserData in JSON is painful to watch these days.

I've also added a technique to split out the UserData logic to it's own log file, as otherwise it goes in cloud-init.log, which is also painful to read.

Alexandre Hamon · Answer 5 · 2020-05-11T12:22:04.347

The proper way to do it is to call secret manager to get your data, here is how I succeded doing it :

SftpCredsUserPasswordSecret:
    Type: 'AWS::SecretsManager::Secret'
    Properties:
      Name: 'sftp-creds-user-password-secret'
      Description: DB Credentials
      GenerateSecretString:
        SecretStringTemplate: '{"username":"sftpuser"}'
        GenerateStringKey: "password"
        PasswordLength: 30
        ExcludePunctuation: true

  Ec2SftpRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument: # Tells that Ec2 can assume this role
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Action:
              - sts:AssumeRole
            Principal:
              Service:
                - ec2.amazonaws.com
      Policies: # Tells that you can call for the secret value
        - PolicyName: "root"
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action: "secretsmanager:GetSecretValue"
                Resource: !Ref SftpCredsUserPasswordSecret
      RoleName: 'role-ec2-sftp'

  Ec2SftpIamInstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      InstanceProfileName:  'instance-profile-ec2-sftp'
      Roles:
        - !Ref Ec2SftpRole

  Ec2Sftp:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: "ami-06ce3edf0cff21f07"
      InstanceType: t2.micro
      SecurityGroupIds:
        - !ImportValue SgBridgeId
        - !ImportValue SgSftpId
      SubnetId: !ImportValue PublicSubnetAZbId
      KeyName: !Ref KeyName
      UserData:
        Fn::Base64: !Sub
        - |
          #!/bin/bash

          # Get Variables
          secret=`aws --region ${AWS::Region} secretsmanager get-secret-value --secret-id ${secretRef} --query SecretString --output text`
          sftpuser=`echo "$secret" | sed -n 's/.*"username":["]*\([^(",})]*\)[",}].*/\1/p'`
          sftppassword=`echo "$secret" | sed -n 's/.*"password":["]*\([^(",})]*\)[",}].*/\1/p'`

          # Create Sftp User
          adduser $sftpuser
          echo "$sftpuser:$sftppassword" | chpasswd

          # Configure sftp connection
          echo "" >> /etc/ssh/sshd_config
          echo "Match User $sftpuser" >> /etc/ssh/sshd_config
          echo "    PasswordAuthentication yes" >> /etc/ssh/sshd_config
          echo "    ForceCommand /usr/libexec/openssh/sftp-server" >> /etc/ssh/sshd_config

          # Restart the service
          systemctl restart sshd
        -
          secretRef: !Ref SftpCredsUserPasswordSecret
      IamInstanceProfile: !Ref Ec2SftpIamInstanceProfile
      Tags:
        - Key: Name
          Value: 'ec2-sftp'

The problem with this solution is that the EC2 instance needs access to the Secrets Manager secret, which is not always the case, nor is it necessarily desired. — Andrew, Nov 28 '22 at 06:01
Sure, You could resolve the secret in 'secretRef' and not have to allow EC2 to read password ! — Alexandre Hamon, Nov 29 '22 at 08:09

score -2 · Answer 6 · answered May 03 '21 at 05:55

-2

AWS CloudFormation enhances the existing dynamic referencing of AWS Systems Manager Parameter Store parameters in CloudFormation templates. You can now reference the latest Systems Manager parameter values in CloudFormation templates without specifying parameter versions.

https://aws.amazon.com/about-aws/whats-new/2021/04/now-reference-latest-aws-systems-manager-parameter-values-in-aws-cloudformation-templates-without-specifying-parameter-versions/

answered May 03 '21 at 05:55

PainPoints

461
8
20

This does not seem to answer the question, which is about expanding dynamic references within the UserData parameter of an EC2 instance or launch config. – kgutwin Nov 09 '21 at 20:34

score -2 · Answer 7 · answered Sep 09 '22 at 03:58

-2

I have a cfn template similar to below working

Parameters:
  ASecret:
    Type: String
    Default: '{{resolve:secretsmanager:ASecret}}'
-
-
-
UserData: 
        Fn::Base64: !Sub |
            #!/bin/bash -xe

            echo ${ASecret}

answered Sep 09 '22 at 03:58

anthos

187
2
2
11

Dynamic References to Specify Secret Manager Values in AWS Cloudformation

7 Answers7

Linked