sns unable to trigger lambda when created using cloudformation

Question

I have gone through many blogs but none of them solves my issue. SNS created by cloudformation is unable to trigger the lambda created by the same cloudformation, I see the trigger as sns in lambda but it didn't trigger it, below is the code.

Tried all solution suggested like using only SourceArn in lambda permission instead of SourceAccountId and all

LambdaBasicExecutionRole:
    Type: "AWS::IAM::Role"
    Properties:
      RoleName: "LambdaBasicExecutionRole"
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          -
            Effect: "Allow"
            Principal:
              Service:
                - "lambda.amazonaws.com"
            Action:
              - "sts:AssumeRole"
      Path: "/"
      Policies: 
        - 
          PolicyName: "LambdaPolicyEC2KeyPair"
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              -
                Effect: "Allow"
                Action: 
                  - "kms:ListGrants"
                  - "kms:CreateGrant"
                  - "kms:Encrypt"
                  - "kms:Decrypt"
                Resource: "arn:aws:kms:*:*:*"
              - 
                Effect: "Allow"
                Action: 
                  - "logs:CreateLogGroup"
                  - "logs:CreateLogStream"
                  - "logs:PutLogEvents"
                Resource: "arn:aws:logs:*:*:*"
              - 
                Effect: "Allow"
                Action: "ec2:CreateKeyPair"
                Resource: "*"
              - 
                Effect: "Allow"
                Action: "ssm:PutParameter"
                Resource: "*"

  LambdaFunctionEC2KeyPair:
    Type: AWS::Lambda::Function
    Properties:
      FunctionName: LambdaFunctionEC2KeyPair
      Description: "Lambda Function to create EC2 KeyPair and storing it's private key securely to paramater store"
      Handler: index.handler
      Runtime: python3.6
      Role: !GetAtt LambdaBasicExecutionRole.Arn
      Code:
        ZipFile: |
          import boto3, os, botocore, cfnresponse

          client = boto3.client('ec2')
          ssm = boto3.client("ssm")

          def handler(event, context):
            ###############################
            # Variable Defination from CF #
            ###############################

            IIS = ['service', 'engine', 'micro']

            namespace = "IIS"
            keyid = os.environ['kmsid']
            env = os.environ['env']

            for iis_tier in IIS:
              keyname = 'IIS-EC2-KeyPair-'+iis_tier+'-'+env
              try:
                response = client.create_key_pair(
                  KeyName=keyname
                )

              except botocore.exceptions.ClientError as e:
                if e.response['Error']['Code'] == 'InvalidKeyPair':
                  print ("Invalid Key Pair Duplicate Error")
                  continue
                else:
                  continue

              try:
                ssm_response = ssm.put_parameter(
                  Name=f"/{namespace}/{env}/EC2-KeyPair/{iis_tier}",
                  Value=response['KeyMaterial'],
                  Type="SecureString",
                  KeyId=keyid,
                  Description='Private key for '+iis_tier+' '+env+' EC2 instance for ssh connection, one would need it for making ssh connection with the instance for administrative purposes'
                )
              except botocore.exceptions.ClientError as e:
                if e.response['Error']['Code'] == 'AccessDeniedException':
                  print ("Access Denied Error")
                  continue
                else:
                  continue
            cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData, physicalResourceId )
            return



      Environment:
        Variables: 
          env: !Ref Environment
          kmsid: !Ref kmsKeyIIS
    DependsOn: LambdaBasicExecutionRole


  EC2KeyPair:
    Type: Custom::EC2KeyPairResource
    Properties:
      ServiceToken: !GetAtt LambdaFunctionEC2KeyPair.Arn

I took your code and inserted it into a CloudFormation template. I had to insert some permissions where it said ``. It successfully created the SNS topic and the Lambda function. I then manually published a message to the SNS topic in the console and it **successfully triggered the Lambda function**. What makes you think that it is not working for you? How are you testing whether the function executed? — John Rotenstein, Nov 08 '19 at 21:30
Thanks @JohnRotenstein for the response, I wanted the trigger to happen automatically when I create the Lambda and SNS. Am i missing something? As whatever resource my lambda is going to create will be used further, so I need to trigger it as part of the cloudformation deployment. — Sid, Nov 11 '19 at 00:13
What do you mean by "trigger to happen automatically when I create the Lambda and SNS"? The configuration you have will add Amazon SNS as a trigger for the Lambda function. Then, whenever a message is sent to the SNS topic, the Lambda function will be triggered, with the message passed to the function. — John Rotenstein, Nov 11 '19 at 00:32
I want an adhoc trigger for the lambda function when it gets created by cloudformation, so i need to a publish a message in order to trigger it, i thought creating sns will trigger it for the first time. Is there a way to trigger only once that too during the lambda function is created? — Sid, Nov 11 '19 at 02:58
Ah! If your only intention is to trigger an AWS Lambda function when the CloudFormation stack is created, then you should be using an [AWS Lambda-backed Custom Resource](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources-lambda.html). Include the Lambda function in the template, plus a Custom Resource that points to the function and it will trigger when the stack deploys. Please note that the Lambda function needs to **signal back** when it is complete. A function library is provided on that page. Any problems, create a new Question. No SNS required. — John Rotenstein, Nov 11 '19 at 04:26
@JohnRotenstein, yeah I figured it that I need a custom resource, but the issue is, in cloudformation stack it doesn't progress, it's kind of hung. `EC2KeyPair: Type: Custom::EC2KeyPairResource Properties: ServiceToken: !Sub ${LambdaFunctionEC2KeyPair.Arn}` — Sid, Nov 11 '19 at 05:40
Your Lambda function needs to specifically signal-back to CloudFormation when it is complete. Otherwise, it will appear to hang and will eventually time-out. See my answer, below. — John Rotenstein, Nov 11 '19 at 06:18
Your code is not using `cfn-response` to signal back to CloudFormation that the custom resource has completed. — John Rotenstein, Nov 11 '19 at 07:06
Hi John, I am still getting the weird cloudformation rollback, Howver lambda is getting triggered and doing what it needs to do, but my cfn-response is not letting the custom resource to pass through to success. — Sid, Nov 18 '19 at 04:50

John Rotenstein · Accepted Answer · 2019-11-19T00:24:34.433

1

It appears that you are wanting to trigger an AWS Lambda function when the CloudFormation stack deploys.

You can do this with an AWS Lambda-backed Custom Resource.

The template should include:

The Lambda function
A Custom:: entry to trigger your Lambda function

The Lambda function will need to signal back when it is complete. There is a cfn-response Module provided to assist with this. It is available for Node.js and Python.

Here is a basic CloudFormation template that deploys and runs a Custom Resource:

AWSTemplateFormatVersion: 2010-09-09

Resources:

  LambdaBasicExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: MyLambdaRole
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          -
            Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

  LambdaFunctionTest:
    Type: AWS::Lambda::Function
    DependsOn: LambdaBasicExecutionRole
    Properties:
      FunctionName: LambdaFunctionTest
      Description: Lambda Function to test that Custom Resource works
      Handler: index.handler
      Runtime: python3.6
      Role: !GetAtt LambdaBasicExecutionRole.Arn
      Code:
        ZipFile: |
          import boto3
          import cfnresponse

          def handler(event, context):
            print('This is in the handler')

            responseData = {}
            cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData)
            return

  CustomFunctionTest:
    Type: Custom::CustomFunctionTest
    Properties:
      ServiceToken: !GetAtt LambdaFunctionTest.Arn

edited Nov 19 '19 at 00:24

answered Nov 11 '19 at 04:37

John Rotenstein

241,921
22
380
470

It's failing, and going into rollback. Are you saying the lambda function needs to have code to send response to cloudformation stack, how do we do it? – Sid Nov 11 '19 at 06:56
1

Sounds like you are not (successfully) signalling back from the Lambda function. Use the `cfn-response` module to perform the signal. – John Rotenstein Nov 11 '19 at 07:03
Thanks John, I could see cfnresponse sends physicalresourceId as well, is there a way I can export in output of cloudformation template for further use, I couldn't find any code for that. Also when you delete a stack, it deletes the custom resource and lambda, how do we delete resources created by lambda? – Sid Nov 12 '19 at 01:07
The `physicalResourceId` is shown as optional, so you probably don't need it. As for deleting, what do you mean by "resources created by Lambda"? If you created resources through API calls from your Lambda code, then CloudFormation would have no knowledge of such resources, so it would not be able to delete them. – John Rotenstein Nov 12 '19 at 01:17
Code which i updated in my question section is giving me weird errors like index module not found, cfnresponse not found. – Sid Nov 12 '19 at 04:36
In looking at a [similar solution](https://github.com/stelligent/cloudformation-custom-resources/blob/master/lambda/python/customresource.py), try using `context.log_stream_name` as the `physicalResourceId`. Or, frankly, try leaving out `physicalResourceId` altogether since it is listed as optional on the API call. – John Rotenstein Nov 12 '19 at 04:47
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/202183/discussion-between-sid-and-john-rotenstein). – Sid Nov 12 '19 at 05:48

Kristijan Iliev · Answer 2 · 2019-11-08T10:42:04.853

I have AWS::Serverless::Function and there's Events attribute in the its properties. If you had that, then your configuration would be something like this:

LambdaFunctionEC2KeyPair:
  Type: AWS::Lambda::Function
  Properties:
    FunctionName: LambdaFunctionEC2KeyPair
    Description: "Lambda Function to create EC2 KeyPair and storing it's private key securely to paramater store"
    Handler: index.handler
    Runtime: python3.6
    Role: !GetAtt LambdaBasicExecutionRole.Arn
    Code:
      ZipFile: |
        My code


    Environment:
      Variables: 
        env: !Ref Environment
        kmsid: !Ref kmsKeyIIS
    Events:
      SNSTopicMessage:
        Type: SNS
        Properties:
          Topic:
            Fn::Join:
              - ':'
              - - arn
                - Ref: AWS::Partition
                - sns
                - Ref: AWS::Region
                - Ref: AWS::AccountId
                - SNSTopicLambdaInvoke
  DependsOn: LambdaBasicExecutionRole

I am checking about your use case having AWS::Lambda::Function.

You can check this example.

About the difference between a Serverless Function, and a Lambda Function you can check in this answer: What is the difference between a Serverless Function, and a Lambda Function

Thanks Kris for the response, what does AWS::Partition refer to, and in your code you meant AWS::Serverless::Function. — Sid, Nov 11 '19 at 00:24
Returns the partition that the resource is in. For standard AWS regions, the partition is `aws`. You can check more on the [official aws documentation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html) — Kristijan Iliev, Nov 11 '19 at 09:13

sns unable to trigger lambda when created using cloudformation

2 Answers2