2

I have recently come across a problem I have never seen or even heard of before. I am using an AWS HTTP API Gateway which is integrated with a simple lambda function that returns a response from a DB. Hardly rocket surgery. The problem is this; when I make a request the first time, the response comes back as it should, with the body as JSON.

enter image description here

When I make an identical request a few seconds later, the response looks like this:

enter image description here

And the third time, this:

enter image description here

It gets worse and worse each time. My function response follows the rules defined in the developer guide, and as you can see from the first response, it's working just fine.

Why it would escape multiple times after that is beyond me. In my lambda code, I stringify the JSON once response.body = JSON.stringify(response.body) to follow the spec. There is nothing in my code that even knows to stringify multiple times based on an individual users IP address only after a certain number of requests within a certain timespan that go back to normal after a half hour or so.

Any help would be appreciated.

UPDATE:

index.js

const users = require ('./handlers/users.js')
const auth = require ('./lib/auth.js')

exports.handler = async (event) => {

  const body = event.body ? JSON.parse(event.body) : undefined;
  const queryString = event.queryStringParameters
  const authHead = event.headers.authorization ? JSON.parse(event.headers.authorization) : undefined
  
  const path = event.requestContext.http.path
  const method = event.requestContext.http.method

  const authUser = auth(authHead)

 // I have more handlers, but you get the idea...

  const handlers = {
    '/users': {
      POST: users.post,
      PATCH: users.patch,
      GET: users.get,
    },
  }

  const response = await handlers[path][method](authUser, queryString, body)
  response.body = JSON.stringify(response.body)

  return response;
};

/handlers/users.js

const AWS = require('aws-sdk')
const docClient = new AWS.DynamoDB.DocumentClient()

const response = {
  statusCode: 500,
  body: {
    message: 'unhandled error',
    payload: undefined
  }
}

exports.get = async (authUser, queryString, body) => {


  const params = {
    TableName: 'db-name',
    Key: {
      'PK': undefined,
      'SK': undefined
    }
  }

  if (queryString.id) {
    params.Key.PK = queryString.id;
    params.Key.SK = `user#${queryString.id}`;
  } else if (queryString.username) {
    params.IndexName = 'GSI'
    params.ExpressionAttributeValues[':ID'] = queryString.username
  } else {
    response.statusCode = 400
    response.body = { message: 'bad request', payload: undefined }
    return response
  }

  try {
    const results = await docClient.get(params).promise()
    response.body.payload = results
    response.statusCode = 200
    response.body.message = 'success'
    return response
  }
  catch (err) {
    console.log(err)
    response.statusCode = 400
    response.body = 'failed'
    return response
  }

}
Tom Parke
  • 119
  • 5
  • Can you show the lambda code, or a reproducible example of one? – Marcin Oct 05 '20 at 06:32
  • @Marcin - Updated the question. Thanks for showing interest. – Tom Parke Oct 05 '20 at 06:42
  • Looks like a reverse proxy on their side stores the result and, when asked again, for some reason quotes it, notices that its hash has changed, and replaces it with the quoted version, rinse and repeat. – yeoman Oct 05 '20 at 06:46
  • 1
    What happens when you add something insignificant to the request URL, like ?bla=5 - this could be a workaround, just using a random / increasing number instead of 5 – yeoman Oct 05 '20 at 06:48
  • @yeoman - I added '&hello' at the end of the URL but it still added an extra set of escapes. I also tried changing the request id and still got yet another extra set of escapes. This would prove that neither the response or request need to be identical to reproduce the problem. – Tom Parke Oct 05 '20 at 06:54
  • Would have been such an easy explanation... – yeoman Oct 05 '20 at 07:50
  • Nothing in your code looks suspicious... – yeoman Oct 05 '20 at 07:57
  • 2
    This only happens with this particular code? What if you go back to basics, and write most simplest function returning constant json, without any database or anything else? – Marcin Oct 05 '20 at 08:42
  • 1
    @Marcin - That was a good idea. I thought my code was clean but when I added the response right at the top before any database call it worked perfectly and continued to work perfectly without extra escape characters. I'll explore this further. – Tom Parke Oct 05 '20 at 09:20
  • @TomParke Glad to hear. If you know the root cause of the issue, you can answer your own question for future reference. – Marcin Oct 05 '20 at 09:21
  • It's looking like this is actually an issue stemming from dynamodb. I logged the results object after an initial successful api call and saw body: { message: 'success', payload: { Item: [Object] } }, and the stringified version was of course expanded. However, the second time I called it, when I received the first set of escaped characters, the log looked instead like this: body: `{"message":"success","payload":{"Item":{"realname":"Tom Parke","location":"Brisb... (it went on for a while) }}}`. The response from the DynamoDB call was now a string instead of an object. I'll keep looking... – Tom Parke Oct 05 '20 at 10:58
  • Hold up. Sorry guys it looks like user202729's answer is actually correct. Lambda is reusing the environment each time the code was run, which meant the global variables were leaking out into subsequent code runs. An article that discusses this: https://www.trek10.com/blog/stateless Incredible. I lost a half day to this. – Tom Parke Oct 05 '20 at 11:06

1 Answers1

2

You defined response as a const global object:

const response = {
  statusCode: 500,
  body: {
    message: 'unhandled error',
    payload: undefined
  }
}

Because in JavaScript, assignment is not object copy, and const object can be modified, when you modify response.body, you're actually modifying the body property of the global response object.

To fix the issue: just move the const response = ... assignment to inside the async (authUser, queryString, body) => { ... } function. It will create a new object each time.

user202729
  • 3,358
  • 3
  • 25
  • 36
  • 1
    Remark: when it "works properly" again, it's likely because the process is restarted. I'm not very familiar with Amazon web service so I don't know what's its policy. And it's not even dependent on the IP address, just the Node.js process that serves the request. – user202729 Oct 05 '20 at 10:50
  • Thanks a lot for that. I had no idea lambda reused the environment along with all global variables on subsequent invocations. For anyone interested in what's going on here: https://www.trek10.com/blog/stateless – Tom Parke Oct 05 '20 at 11:07