Cannot Validate Twilio Request in an AWS Lambda Custom Authorizer

Question

I've built a custom authorizer lambda function with NodeJS, which I've configured to authorize another lambda function in AWS. This other function is triggered from an HTTP endpoint, and has the URL that I configured in my Twilio Messaging Service as the webhook URL with a GET method.

I have to use GET, because AWS does not include a POST request's headers into the input param of the authorizer function, and I need the header in order to get the X-Twilio-Signature.

In the authorizer function, I'm invoking the Twilio Node Helper validateRequest(token, signature, url, params) function and providing my auth token, the Twilio signature from the request header, and the exact same URL as configured in the webhook (no query params, no frills, just an https url with a path to the api resource).

However, the params is where I think things are breaking down and why the validation fails.

Since I'm using a GET method for the webhook, does that mean that when Twilio created the signature hash on their end, there was no POST data appended (per their docs on https://www.twilio.com/docs/api/security), or should I provide all the form data which they provide in the querysting of my GET request??

No matter what I've tried, my validation keeps failing as if the params I'm using are different than what Twilio did to create the signature.

I've created a simple test to see if I can validate the request using the params and signature of an actual HTTP request I made, but it never seems to work. Here's my simple test:

const token = '[my auth token]';
const url = 'https://my-api.company.io/sms/receive';
const signature = '[twilio header signature]';

const params = { 
  MessagingServiceSid: '[sid to my msg svc]',
  ApiVersion: '2010-04-01',
  SmsSid: 'SM6b3e14ea5e87ff967adb0c00c81406b8',
  SmsStatus: 'received',
  SmsMessageSid: 'SM6b3e14ea5e87ff967adb0c00c81406b8',
  NumSegments: '1',
  ToState: 'TX',
  From: '+19998675309',
  MessageSid: 'SM6b3e14ea5e87ff967adb0c00c81406b8',
  AccountSid: '[my account sid]',
  ToZip: '75229',
  ToCity: 'DALLAS',
  FromCountry: 'US',
  FromCity: 'IRVING',
  To: '[my twilio number]',
  FromZip: '75014',
  ToCountry: 'US',
  Body: 'Super duper',
  NumMedia: '0',
  FromState: 'TX' 
};
const result = twilio.validateRequest(token, signature, url, params);
console.log(result);

UPDATE To respond to an answer from Phil (Twilio Dev Evangelist), here's what I see in the logs from my authorizer function when I switch to using a POST webhook URL (this wouldn't fit in a comment, so I'm editing the Q).

Note that this payload does not have any of the above mentioned parameters which are provided by Twilio in the body of the POST request and which I'd presumably need to provide to the twilio.validateRequest function:

{
  type: 'REQUEST',
  methodArn: 'arn:aws:execute-api:us-east-1:********:********/dev/POST/receive',
  resource: '/receive',
  path: '/sms/receive',
  httpMethod: 'POST',
  headers: {
    Accept: '*/*',
    'CloudFront-Viewer-Country': 'US',
    'CloudFront-Forwarded-Proto': 'https',
    'CloudFront-Is-Tablet-Viewer': 'false',
    'CloudFront-Is-Mobile-Viewer': 'false',
    'User-Agent': 'TwilioProxy/1.1',
    'X-Forwarded-Proto': 'https',
    'CloudFront-Is-SmartTV-Viewer': 'false',
    Host: 'api.myredactedcompany.io',
    'X-Forwarded-Port': '443',
    'X-Amzn-Trace-Id': 'Root=**************',
    Via: '1.1 ***************.cloudfront.net (CloudFront)',
    'Cache-Control': 'max-age=259200',
    'X-Twilio-Signature': '***************************',
    'X-Amz-Cf-Id': '****************************',
    'X-Forwarded-For': '[redacted IP addresses]',
    'Content-Length': '492',
    'CloudFront-Is-Desktop-Viewer': 'true',
    'Content-Type': 'application/x-www-form-urlencoded'
  },
  queryStringParameters: {},
  pathParameters: {},
  stageVariables: {},
  requestContext: {
    path: '/sms/receive',
    accountId: '************',
    resourceId: '*****',
    stage: 'dev',
    requestId: '5458adda-ce2c-11e7-ba08-b7e69bc7c01c',
    identity: {
      cognitoIdentityPoolId: null,
      accountId: null,
      cognitoIdentityId: null,
      caller: null,
      apiKey: '',
      sourceIp: '[redacted IP]',
      accessKey: null,
      cognitoAuthenticationType: null,
      cognitoAuthenticationProvider: null,
      userArn: null,
      userAgent: 'TwilioProxy/1.1',
      user: null
    },
    resourcePath: '/receive',
    httpMethod: 'POST',
    apiId: '*******'
  }
}

From the docs, it appears you do not need to use the parameters *again* because they are already in the URL -- but you need the *complete* URL, with query string, not just the path. POST request signing appends the POST parameters, sorted, to the end of the URL, but it wouldn't make sense for them to be included "again" when signing a GET. — Michael - sqlbot, Nov 18 '17 at 22:48
@Michael-sqlbot as I mentioned in the OP, the URL I provide in the Twilio dashboard has no querystring params. I am NOT using POST due to a limitation with AWS lambda authorizer funcs not actually getting passed the original request's POST form data in the event. However, they do make available the original request's Querystring, which is how I am able to get the values from the Twilio callback (e.g. the "params" in the code above). — Thiago Silva, Nov 20 '17 at 03:20
@Michael-sqlbot I have tried calling the validateRequest method with the url exactly as I have set up in the Twilio dashboard, and with an empty object for the params, as well as with the object listed above in the code, which is the params object that Twilio sent in the request. Neither have worked thus far. — Thiago Silva, Nov 20 '17 at 03:27
Your `const url` ... I don't think that's the right URL... I'm suggesting you need to use the actual, complete, final URL -- what you are *receiving* from them on the incoming request, and an empty params object. Unfortunately, their docs example doesn't show an example of this case. — Michael - sqlbot, Nov 20 '17 at 04:30
@Michael-sqlbot Unfortunately, I don't get the "final" URL as an input param to the authorizer function (see actual payload in update above). I can attempt to build the URL with the querystring params as Twilio would have sent, but that's arbitrary since I don't know which order they tacked on each parameter in the querystring. — Thiago Silva, Nov 20 '17 at 20:41

philnash · Accepted Answer · 2017-11-21T00:43:45.090

Twilio developer evangelist here.

The issue here is that query string parameters are treated differently to POST body parameters when generating the signature.

Notably part 3 of the steps used to generate the request signature says:

If your request is a POST, Twilio takes all the POST fields, sorts them by alphabetically by their name, and concatenates the parameter name and value to the end of the URL (with no delimiter).

(Emphasis mine.)

This means that if you are trying to reconstruct the original URL, you will need to reconstruct the original query string with &s and =. The difficulty I can see here is that you don't know the original order of the parameters and I don't know if the order is not arbitrary.

Issue number 2 is that the Request authorizer will not send the POST body to a Lambda function.

So, either way you try to work it, a custom authorizer will never get all the details that the Twilio request validator requires to validate the request.

My only advice now is to move away from using the authorizers and just build request validation into your final Lambda function. It's not as nice as separating the concerns of validating the request and responding to the request, but since custom authorizers do not support all the features required, it's the only thing I can think of right now.

Now, the other thing that caught my eye was saying that you couldn't get the headers in a POST request authorizer. I had a look around and this Stack Overflow answer suggests it is now possible (only since September) to receive all the headers to a POST request to a custom authorizer, as long as you use the Request type authorizer and not a Token authorizer.

~~So my advice would be to change over to a Request authorizer, a POST request webhook from Twilio and the code you already have should work.~~

Let me know if that helps at all.

Thanks, Phil. I am using the `Request` authorizer, not `Token`, so I can access the Querystring in the function. I don't have a problem getting the request headers, as that is where the signature is found. The problem lies in getting the form data when sent with a POST method. AWS still doesn't provide that data into the authorizer. From the AWS link you provided, scroll till you find the code below the following text and you will see the exact JSON passed to the func: **The following example shows an input to a REQUEST authorizer for an API method (GET /request) with a proxy integration** — Thiago Silva, Nov 20 '17 at 16:38
Phil, I've edited my original Q to provide details on what happens when I switch to a POST webhook (I already used a Request authorizer). Please read above for details. — Thiago Silva, Nov 20 '17 at 20:08
Apologies, I thought you said you couldn't get the headers from a POST request, not the body. I'll have a further look. — philnash, Nov 21 '17 at 00:32
Hey Thiago, I've looked into it again and to me it appears that neither format or request method supports all the features required for the Twilio request validator to work, so I recommend just moving the validation to your main Lambda function. I updated my answer with this information. — philnash, Nov 21 '17 at 00:44

Cannot Validate Twilio Request in an AWS Lambda Custom Authorizer

1 Answers1

Linked