Formatting DynamoDB data to normal JSON in API gateway

Question

just like the question asked here

I do a query or scan operation on dynamoDB using dynamoDB proxy service on AWS API Gateway to read data for the Client and I get DynamoDB formatted JSON data in reply.

Although I can use the "Method Response" to convert but when the Data is above 1000 records - I cannot handle it due to the limitation of foreach loop in Method Response.

Is there a flag or a setting somewhere in dynamodb or in api gateway such that I get normal json rather than the dynamoDB formatted JSON ?

DynamoDB Formatted JSON example

{
  "videos": [
    {
      "file": {
        "S": "file1.mp4"
      },
      "id": {
        "S": "1"
      },
      "canvas": {
        "S": "This is Canvas1"
      }
    },
    {
      "file": {
        "S": "main.mp4"
      },
      "id": {
        "S": "0"
      },
      "canvas": {
        "S": "this is a canvas"
      }
    }
  ]
}

Normal Json example of the same

{
  "videos": [
    {
      "file": "file1.mp4",
      "id": "1",
      "canvas": "This is Canvas1"
    },
    {
      "file": main.mp4",
      "id": "0",
      "canvas": "this is a canvas"
    }
  ]
}

score 2 · Answer 1 · answered Mar 28 '19 at 20:53

2

Using an Integration Response mapper!

The mapper would probably look something like: (I'm using your provided response)

#set($inputRoot = $input.path('$'))
{
  "videos": [
#foreach($elem in $inputRoot.Items)
  { "file": "$elem.file.S",
    "id": "$elem.id.S",
    "canvas": "$elem.canvas.S" }#if($foreach.hasNext), #end
#end 
  ] 
}

Please consult the linked documentation for more complete guidance on attaching AWS Services directly to API Gateway.

answered Mar 28 '19 at 20:53

Jon Bristow

1,675
3
27
42

Whoops! I didn't notice the 1000 rows limitation! – Jon Bristow Mar 28 '19 at 21:00
So does Integration Response have a similar 1000 foreach loop limit? – cellepo Mar 25 '20 at 09:32

score 1 · Accepted Answer · answered Sep 07 '18 at 06:48

1

The possible solution for this is using a aws lambda to query/scan dynamoDb and reply back the domain object after converting it into a json using some json lib, like GSON or jackson.

If any one has knowledge about an alternate option - please do post here. Thanks

answered Sep 07 '18 at 06:48

Abdeali Chandanwala

8,449
6
31
45

1

Commenting here because I was too quick and didn't completely parse your question. According to https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html a foreach will only execute 1000 times. This is a hard cap. – Jon Bristow Mar 28 '19 at 21:02
Found viable solution here: [link](https://stackoverflow.com/a/65757215/7813744) – LANimal Aug 10 '23 at 09:01

Formatting DynamoDB data to normal JSON in API gateway

2 Answers2