3

I have been experimenting with AWS API Gateway and AWS Lambda to try out a serverless architecture. Have been going through blogs and AWS documentation. Have tried out sample GET/POST. But, I have the following requirement w.r.t tracking user events from my custom application

  • Events are posted from my application to API end point
    • I wanted the API to respond back with a custom response (Say {'fine'}) (acknowledging that the request has been received)
  • After the response is sent, hand over the event payload to a AWS Lambda function

As per the documentation, I understand, a) I can post events to API end point b) On GET/POST trigger an AWS Lambda Function - Respond back from AWS Lambda function to API request

I wanted to change the above and modify it to a) Post events to API end point a.0) Respond back acknowledging that request is received [Say {'fine'} ] b) Trigger AWS Lambda function to process the event payload

Please share across suggestions on how to achieve the same.

user1652054
  • 445
  • 2
  • 11
  • 23

3 Answers3

0

If you want a fast response from the API and not have to wait for the processing of data, you could:

  • post an event to an API Gateway endpoint
  • trigger an AWS Lambda Function A
  • call asynchronously a Lambda Function B using the AWS SDK in the Lambda Function A
  • Call context.succeed() or context.done() or the callback function in the Lambda Function A so it respond back to API Gateway
  • the Lambda Function B can process the data while API Gateway already received a response
Alexis N-o
  • 3,954
  • 26
  • 34
  • Thanks. This could mean double billing though. I was wondering if there is a way to let API gateway close the response loop (Like how it is possible in Nodejs express app end points) and then continue processing. – user1652054 May 19 '16 at 06:47
  • Sure, you would be charged for this extra lambda execution, but with 128MB of memory and an execution probably inferior to 100ms, the extra cost should be very limited. You'd have to compare it with the price of other solutions like Kinesis, depending on the volume of requests your expect. – Alexis N-o May 19 '16 at 13:51
0

Another asynchronous model many customers have used:

  1. Set up an API configured to send requests to Amazon Kinesis. This API could acknowledge the request.
  2. Set up AWS Lambda to consume your Kinesis stream.

This setup has some advantages for high workload APIs as fetches from the Kinesis stream can be batched and don't require a 1-to-1 scaling of both your API Gateway limits and Lambda limits.

Update

To answer your questions about scalability:

Kinesis

Kinesis scales by adding what it calls "shards" to the stream. Each shard handles a portion of your traffic, based on a partition key. Each shard scales up to 1000 rps or 1MBps (see limits). Even with the lower default 25 shards, this would support up to 25,000 rps or 25MBps with an evenly distributed partition key.

API Gateway

API Gateway has a default account level limit of 500 rps, but this can easily be extended by requesting a limit increase. We have customers in production that are using the service at limits above your current suggested scale.

Bob Kinney
  • 8,870
  • 1
  • 27
  • 35
  • Sure. Will try this out. – user1652054 May 19 '16 at 06:48
  • Online reading suggests that Kinesis might not scale well for very high- throughput like Kafka. Say, if I have scale up this for 6000 - 8000 requests per second, Would API Gateway & Kinesis be able to handle this? Say my avg. payload size is around 25 - 30 KB? – user1652054 May 19 '16 at 10:37
  • @user1652054 I updated my answer with some info about scaling. Hopefully this answers your question. – Bob Kinney May 19 '16 at 15:29
  • Sure. Thanks that was helpful. I will get started on trying out a few scenarios – user1652054 May 20 '16 at 05:54
0

You should first run some tests to see what type of real world response times you are getting from having your lambda function complete all the logic. If the times are above what you feel are acceptable for your use case, here is another asynchronous solution utilizing an SNS Topic to trigger a secondary Lambda function.

  1. Client Request to API Gateway -> Calls Lambda function A
  2. Lambda A verifies payload and then publishes to SNS Topic X
  3. Lambda A returns {fine} success message -> API Gateway -> client
  4. SNS Topic X triggers Lambda function B
  5. Lambda function B implements given logic
Community
  • 1
  • 1
Mikelax
  • 572
  • 3
  • 10