12

I was recently acquainted to AWS's most peculiar syntax of API Gateway Mapping Templates that don't really make much sense to me.

I tried to read the documentation it was a little bit convoluted to say at least https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-mapping-template-reference.html#util-template-reference

What I want to do is very simple. I have a request body in JSON to which I'd like to add additional field timeReceived with $context.requestTime as its value. My naive first try was #set( $bodyObj.timeReceived = $context.requestTime ) which didn't work.

I'm positive that I will get it to work but it will take time and effort to do such simple thing which IMO is silly and AWS should improve the existing documentation.

EDIT: the solution I've written below works, but I'd advocate reconsidering using API Gateway. I recently updated my app to use edge lambda and CloudFront instead which was a much better solution in every way. Some tricky configuration with the lambda not being able to use environment variables hence automating the DeliveryStreamName to the code was a bit problematic (I ended up using custom resolver with Sceptre). But still a lot better. And no need for these silly hacks =).

TeemuK
  • 2,095
  • 1
  • 18
  • 17

2 Answers2

13

EDIT: Well as of writing this thing I found out that 1# I didn't deploy my changes, embarrassingly and 2# It worked. Now I just have to stringify the JSON back with the added field.

10 minutes later: Well that was hard... So instead I found this which worked:

#set($payload = $util.parseJson($input.json('$')))
#set($body =  "{
  #foreach ($mapEntry in $payload.entrySet())
    ""$mapEntry.key"": ""$mapEntry.value"",
  #end
  ""timeReceived"": ""$context.requestTime"",
  ""x-client-ip"": ""$context.identity.sourceIp""
}")

But which expanded the JSON to multiple lines (the JSON is sent to Firehose which then stores it to S3) which is no good.

20 minutes later: My senior dev laughed at me when I told him what I was doing and commented API Gateway Mapping Templates with couple expletives.

40 minutes later: I knew what I had to do: replace all newline characters with empty strings but that proved to be harder than I thought. Also I found out that my nested values inside the object weren't stringified.

50 minutes later: Well actually I can omit the newline replacement by just having the whole foreach-loop in single line. But again I have to somehow stringify the nested values. And the requestTime is formatted in some stupid english format (no offense! :))

70 minutes later: Formatting correctly the nested values is really difficult. I found another example which worked:

#set($body =  "{
  #foreach ($mapEntry in $payload.entrySet())
    #if ($mapEntry.value.size() > 0)
      ""$mapEntry.key"": {
      #foreach($subEntry in $mapEntry.value.entrySet())
        ""$subEntry.key"": ""$subEntry.value""#if($foreach.hasNext),#end
      #end
      },
    #else
      ""$mapEntry.key"": ""$mapEntry.value"",
    #end
  #end
  ""timeReceived"": $context.requestTimeEpoch}
}")

Which translates to if you don't want new lines or extra spaces into this:

#set($body = "{#foreach ($mapEntry in $payload.entrySet())#if($mapEntry.value.size() > 0)""$mapEntry.key"": { #foreach($subEntry in $mapEntry.value.entrySet())""$subEntry.key"": ""$subEntry.value""#if($foreach.hasNext), #end#end }, #else""$mapEntry.key"": ""$mapEntry.value"", #end#end""timeReceived"": $context.requestTimeEpoch }" )

120 minutes spent both debugging and writing this: Well that was a journey. I guess I already answered my own question so hopefully now someone can avoid spending so much time debugging this as I did.

130 minutes later: Aaand now I have to solve the tragedy of adding that string into my CloudFormation template. Oh boy. (Which wasn't even hard, the hard part is redeploying the API gateway which as of writing this I had to do manually from the console)

TeemuK
  • 2,095
  • 1
  • 18
  • 17
  • 4
    The "most peculiar syntax" is [VTL](http://velocity.apache.org/engine/1.7/vtl-reference.html). Adding `##` at the end of each line (a comment) should have the side effect of swallowing the newline at the end of each line, allowing a template that is human readable that still produces output without line breaks. – Michael - sqlbot Jan 19 '18 at 12:52
  • The mapping template works fine for nested JSON as well! However, a number value in the JSON will be surrounded by double-quotes. Any method to ignore double quotes for a number value? – Anush Surendran Dec 04 '19 at 09:25
  • I solved this by concatenating the new field to the JSON string – Anush Surendran Dec 05 '19 at 12:43
  • Doesn't work if you have an Array of json objects as sub values. – pnhegde Sep 30 '20 at 08:26
0

this may also work

#set($inputRoot = $input.path('$'))
{
    ## new field
    "timeReceived": $context.requestTimeEpoch,

    ## add remaining fields back
    #foreach($mapEntry in $inputRoot.entrySet())
        
            #set($mapEntryKeyJsonPath = '$.' + $mapEntry.key)
            "$mapEntry.key": $input.json($mapEntryKeyJsonPath)  
        
        #if($foreach.hasNext),#end
    #end
}
indybee
  • 1,507
  • 13
  • 17