0

I'm trying to implement a Custom Resource for the first time. Whenever I try to deploy the containing stack, creation of the Custom Resource fails with "Response is not valid JSON", but I'm not sure why.

My Custom Resource code is below:

import json
import os
import requests

import logging
# https://stackoverflow.com/questions/37703609
log = logging.getLogger(__name__)
log.setLevel(logging.DEBUG)


def handler(event, context):
    try:
        log.debug(f'DEBUG - called!')
        log.info(f'Event: {event}')
        if event['RequestType'] in ['Create', 'Update']:
          pass
          # I _will_ have logic here eventually - when this is working!
        else:
            log.info('Non-create/update RequestType')

        # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/crpg-ref-responses.html
        responseData = dict(
            {
                'Status': 'SUCCESS',
                'PhysicalResourceId': event['PhysicalResourceId']
                    if 'PhysicalResourceId' in event
                    else context.log_stream_name
            },
            **{key: event[key] for key in
               ['StackId', 'RequestId', 'LogicalResourceId']}
        )
        log.debug(f'Response data: {responseData}')
        requests.put(event['ResponseURL'], data=responseData)
    except Exception as e:
        log.error(f'Lambda failed! {e}')
        # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/crpg-ref-responses.html
        responseData = dict(
            {
                'Status': 'FAILED',
                'Reason': f'See logs in {context["logStreamName"]}',
                'PhysicalResourceId': event['PhysicalResourceId']
                    if 'PhysicalResourceId' in event
                    else context.log_stream_name,
            },
            **{key: event.get(key, '') for key in
               ['StackId', 'RequestId', 'LogicalResourceId']}
        )
        log.debug(f'Response data: {responseData}')
        requests.put(event['ResponseURL'], data=responseData)

And some example log messages from a Create/Delete loop are below:

START RequestId: f151f270-acc6-4cff-b4c0-0eafaf4e5fef Version: $LATEST
[DEBUG] 2021-03-22T19:29:09.33Z f151f270-acc6-4cff-b4c0-0eafaf4e5fef    DEBUG - called!
[INFO]  2021-03-22T19:29:09.33Z f151f270-acc6-4cff-b4c0-0eafaf4e5fef    Event: {'RequestType': 'Create', 'ServiceToken': 'arn:aws:lambda:us-east-1:119281758091:function:prod-stage-ApplicationSta-FetchCommitHistoryFuncti-T7Z15V6TDZSO', 'ResponseURL': 'https://cloudformation-custom-resource-response-useast1.s3.amazonaws.com/arn%3Aaws%3Acloudformation%3Aus-east-1%3A119281758091%3Astack/prod-stage-ApplicationStack/94d24fa0-8b44-11eb-a285-0ec6977a61f1%7CFetchCommitsCustomResource%7C1d80a8f0-f72c-4e50-8039-6c0509dd24ee?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20210322T192907Z&X-Amz-SignedHeaders=host&X-Amz-Expires=7200&X-Amz-Credential=AKIA6L7Q4OWT3GW5BT7K%2F20210322%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=4cc0d2590330f42e8928bdfc2bf5a30c31bfb6228d681e11acf25ef17b8dde2c', 'StackId': 'arn:aws:cloudformation:us-east-1:119281758091:stack/prod-stage-ApplicationStack/94d24fa0-8b44-11eb-a285-0ec6977a61f1', 'RequestId': '1d80a8f0-f72c-4e50-8039-6c0509dd24ee', 'LogicalResourceId': 'FetchCommitsCustomResource', 'ResourceType': 'AWS::CloudFormation::CustomResource', 'ResourceProperties': {'ServiceToken': 'arn:aws:lambda:us-east-1:119281758091:function:prod-stage-ApplicationSta-FetchCommitHistoryFuncti-T7Z15V6TDZSO'}}
[DEBUG] 2021-03-22T19:29:09.293Z    f151f270-acc6-4cff-b4c0-0eafaf4e5fef    Response data: {'Status': 'SUCCESS', 'PhysicalResourceId': '2021/03/22/[$LATEST]8801d1d794c04d25ad31fa43223cbe93', 'StackId': 'arn:aws:cloudformation:us-east-1:119281758091:stack/prod-stage-ApplicationStack/94d24fa0-8b44-11eb-a285-0ec6977a61f1', 'RequestId': '1d80a8f0-f72c-4e50-8039-6c0509dd24ee', 'LogicalResourceId': 'FetchCommitsCustomResource'}
END RequestId: f151f270-acc6-4cff-b4c0-0eafaf4e5fef
REPORT RequestId: f151f270-acc6-4cff-b4c0-0eafaf4e5fef  Duration: 444.73 ms Billed Duration: 445 ms Memory Size: 128 MB Max Memory Used: 61 MB  Init Duration: 463.55 ms    
START RequestId: 616f7c98-1d76-4459-96d3-6257c5620a99 Version: $LATEST
[DEBUG] 2021-03-22T19:29:20.640Z    616f7c98-1d76-4459-96d3-6257c5620a99    DEBUG - called!
[INFO]  2021-03-22T19:29:20.640Z    616f7c98-1d76-4459-96d3-6257c5620a99    Event: {'RequestType': 'Delete', 'ServiceToken': 'arn:aws:lambda:us-east-1:119281758091:function:prod-stage-ApplicationSta-FetchCommitHistoryFuncti-T7Z15V6TDZSO', 'ResponseURL': 'https://cloudformation-custom-resource-response-useast1.s3.amazonaws.com/arn%3Aaws%3Acloudformation%3Aus-east-1%3A119281758091%3Astack/prod-stage-ApplicationStack/94d24fa0-8b44-11eb-a285-0ec6977a61f1%7CFetchCommitsCustomResource%7C85da252d-1d0d-4fe5-84eb-755f1e6dc646?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20210322T192920Z&X-Amz-SignedHeaders=host&X-Amz-Expires=7200&X-Amz-Credential=AKIA6L7Q4OWT3GW5BT7K%2F20210322%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=0bd2e0a28783980b9014289cd852bf02252cdf5cd11f4405d00707002e6e39e9', 'StackId': 'arn:aws:cloudformation:us-east-1:119281758091:stack/prod-stage-ApplicationStack/94d24fa0-8b44-11eb-a285-0ec6977a61f1', 'RequestId': '85da252d-1d0d-4fe5-84eb-755f1e6dc646', 'LogicalResourceId': 'FetchCommitsCustomResource', 'PhysicalResourceId': 'prod-stage-ApplicationStack-FetchCommitsCustomResource-1VTI0Q0VKIJDM', 'ResourceType': 'AWS::CloudFormation::CustomResource', 'ResourceProperties': {'ServiceToken': 'arn:aws:lambda:us-east-1:119281758091:function:prod-stage-ApplicationSta-FetchCommitHistoryFuncti-T7Z15V6TDZSO'}}
[INFO]  2021-03-22T19:29:20.640Z    616f7c98-1d76-4459-96d3-6257c5620a99    Non-create/update RequestType
[DEBUG] 2021-03-22T19:29:20.640Z    616f7c98-1d76-4459-96d3-6257c5620a99    Response data: {'Status': 'SUCCESS', 'PhysicalResourceId': 'prod-stage-ApplicationStack-FetchCommitsCustomResource-1VTI0Q0VKIJDM', 'StackId': 'arn:aws:cloudformation:us-east-1:119281758091:stack/prod-stage-ApplicationStack/94d24fa0-8b44-11eb-a285-0ec6977a61f1', 'RequestId': '85da252d-1d0d-4fe5-84eb-755f1e6dc646', 'LogicalResourceId': 'FetchCommitsCustomResource'}
END RequestId: 616f7c98-1d76-4459-96d3-6257c5620a99
REPORT RequestId: 616f7c98-1d76-4459-96d3-6257c5620a99  Duration: 174.97 ms Billed Duration: 175 ms Memory Size: 128 MB Max Memory Used: 61 MB  

The JSON looks valid to me (in particular, PhysicalResourceId is non-empty, which was an issue when I was first fetching it with just event['PhysicalResourceId']. What's the issue?

scubbo
  • 4,969
  • 7
  • 40
  • 71

1 Answers1

0

I think you may need to convert the dictionary into a json string in your requests.put.

Try something like:

requests.put(event['ResponseURL'], data=json.dumps(responseData))
JD D
  • 7,398
  • 2
  • 34
  • 53
  • Hmm - `requests`' docs for `put` specify `:param data: (optional) Dictionary, list of tuples, bytes, or file-like object to send in the body of the :class:\`Request\`.`, but I'll try it! – scubbo Mar 22 '21 at 23:10
  • Interestingly, stringifying the payload _did_ fix the issue - but, after a little digging around, I found that that was actually because passing a json-string as payload causes `requests` not to add a `'Content-Type': 'application/x-www-form-urlencoded'` header. CloudFormation CustomResource confirmation apparently fails when passed a Content-Type header – scubbo Mar 23 '21 at 02:27
  • Another way would be to use the ‘json’ argument instead of ‘data’. I believe this does the json dump implicitly and also adds a json content type header – JD D Mar 23 '21 at 11:08