Serverless express app terminates before logic within route has completed

Question

I have a simple Express app, hosted on AWS, using the Serverless framework.

I'm using serverless-http to wrap the express app for deployment to an AWS lambda function, and express-async-await to allow the use of async functions for routes.

For one of my routes I want to return a page to the user then continue performing various async tasks. Here is an abbreviated version of my code:

const serverless = require("serverless-http");
const express = require("express");
const mustacheExpress = require("mustache-express");
const aa = require("express-async-await");

// configure our express app
const app = aa(express());
app.use(express.static("static"));
app.engine("html", mustacheExpress());
app.set("view engine", "html");
app.set("views", __dirname + "/views");
app.get("/", async (request, response) => {

  response.render("generating", {  });

  // various async tasks
  await s3.putObject({ });
  await s3.putObject({ });

});

module.exports.handler = serverless(app);

However, from inspecting the logs I can see that the tasks that follow the response.render stop mid-way. The lambda returns without an error, no timeouts, but the tasks are stopped.

Moving response.render to the end of the route means that the async tasks are executed, however, the user sees a spinner rather than a rendered HTML holding pages.

From reading around, I've found numerous references that indicate Express is more than happy to continue execution after a response has been returned.

Any ideas what is going wrong in my case?

score 0 · Answer 1 · answered Oct 04 '20 at 06:53

I would say that the AWS lambda function is terminating the execution after the response is written.

¿Are you using promises with S3? Shouldn't the code to upload be:

await s3.putObject({ }).promise();

Try that. However I would also suggest moving the response.render to the end of the route and handling things from the front-end. For example:

// show spinner

// make AJAX request and handle the result asynchronously, for example, assuming you are using axios:
axios.get(url).then(data => {
  // show result to the user
})

// hide spinner and show other HTML while the request finishes

score 0 · Accepted Answer · answered Oct 06 '20 at 20:56

As far as I know this use case is not presently possible with Lambda and API Gateway, as the invocation to Lambda from API Gateway is of type RequestResponse, and not of type Async. Reference

Once your code (which in this case I believe is serverless-express) calls context.done(), the response is sent from Lambda to API Gateway and the execution context of the Lambda function is finished (either frozen, to be re-used in the next invocation, or recycled into Lambda's underlying infrastructure).

There are several options to invoke a new function to continue processing, so your API can return to the user:

Publish a message to an SNS topic and/or an SQS queue and build a second lambda to process the messages
Use Step Functions
Use the AWS-SDK to directly invoke a second function

Serverless express app terminates before logic within route has completed

2 Answers2