How to parse request body from Amazon SNS Notification in rails

Question

I'm setting up an endpoint in my rails 3.0 app to receive pushed notifications from an Amazon SNS service.

The request that is posted by Amazon has a JSON payload, but they set content-type on the request as "text/plain", which results in Rails not parsing out the body.

Example post request from Amazon's docs:

POST / HTTP/1.1
x-amz-sns-message-type: Notification
x-amz-sns-message-id: 22b80b92-fdea-4c2c-8f9d-bdfb0c7bf324
x-amz-sns-topic-arn: arn:aws:sns:us-west-2:123456789012:MyTopic
x-amz-sns-subscription-arn: arn:aws:sns:us-west-2:123456789012:MyTopic:c9135db0-26c4-47ec-8998-413945fb5a96
Content-Length: 773
Content-Type: text/plain; charset=UTF-8
Host: myhost.example.com
Connection: Keep-Alive
User-Agent: Amazon Simple Notification Service Agent

{
  "Type" : "Notification",
  "MessageId" : "22b80b92-fdea-4c2c-8f9d-bdfb0c7bf324",
  "TopicArn" : "arn:aws:sns:us-west-2:123456789012:MyTopic",
  "Subject" : "My First Message",
  "Message" : "Hello world!",
  "Timestamp" : "2012-05-02T00:54:06.655Z",
  "SignatureVersion" : "1",
  "Signature" : "EXAMPLEw6JRNwm1LFQL4ICB0bnXrdB8ClRMTQFGBqwLpGbM78tJ4etTwC5zU7O3tS6tGpey3ejedNdOJ+1fkIp9F2/LmNVKb5aFlYq+9rk9ZiPph5YlLmWsDcyC5T+Sy9/umic5S0UQc2PEtgdpVBahwNOdMW4JPwk0kAJJztnc=",
  "SigningCertURL" : "https://sns.us-west-2.amazonaws.com/SimpleNotificationService-f3ecfb7224c7233fe7bb5f59f96de52f.pem",
  "UnsubscribeURL" : "https://sns.us-west-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-west-2:123456789012:MyTopic:c9135db0-26c4-47ec-8998-413945fb5a96"
}

So in my controller I can use the request object and read request.body and parse it out myself, like so:

def receive_notification
  if request.content_type =~ /text\/plain/
    body = request.body.read.force_encoding("UTF-8")
    params.merge(JSON.parse(body))
    request.body.rewind
  end
  # ... go on with rest of controller stuff
end

Anyone got a better way to do this? Can we move it back up the chain so my controller isn't worrying about the request object? Is it a bad idea to write a middleware that runs before the parameter parsing and recognizes that a request is from Amazon (by User-Agent, or those custom headers), and changes the content type to "application/json"? (And how do you do that?)

score 3 · Answer 1 · answered Aug 05 '16 at 11:40

I recently came across this issue. Most of the approaches mentioned on the web didn't work for me so I created a middleware. The middleware detects if the message comes from SNS (by looking for the type header) and forces the content_type to application/json.

In my case the message parameter was also JSON passed as a string (SNS message from an S3 bucket notification). This doesn't handle that case but could be extended to do so pretty easily.

# app/middleware/sns_content_type.rb

class SnsContentType
  def initialize(app, message = "Response Time")
    @app = app
  end

  def call(env)
    env['CONTENT_TYPE'] = 'application/json' if env['HTTP_X_AMZ_SNS_MESSAGE_TYPE'].present?
    @app.call(env)
  end
end

Once you've created the middleware you need to install it like so:

# config/application.rb

config.middleware.insert_before ActionDispatch::ParamsParser, "SnsContentType"

This inserts the middleware just before the parameters are parsed meaning that the params parser will see the JSON content type.

Is there no way to do this on a controller (or even per-action) basis? For one, to keep the change localized to only where it's needed. And also to keep yet another piece of middleware out of the stack. Very few requests will ever need this, compared to the many-many requests that don't. — stevenharman, Oct 04 '17 at 16:43
The problem I had trying to do it at the controller level was that you need to place this code before rails parses the request in order to have it interpret the json. If you want to keep things fully in the controller you'd have to use something like `JSON.parse(request.raw)` and do things yourself. — Steve Smith, Oct 19 '17 at 10:47

score 0 · Answer 2 · answered Mar 27 '23 at 18:27

In Q1 2023, Amazon SNS launched support for custom Content-Type headers for HTTP messages delivered from topics. Here's the launch post: https://aws.amazon.com/about-aws/whats-new/2023/03/amazon-sns-content-type-request-headers-http-s-notifications/

You'll have to modify the DeliveryPolicy attribute of your Amazon SNS subscription, setting the headerContentType property to application/json, or any other value supported. You can find all values supported here: https://docs.aws.amazon.com/sns/latest/dg/sns-message-delivery-retries.html#creating-delivery-policy

{
    "healthyRetryPolicy": {
        "minDelayTarget": 1,
        "maxDelayTarget": 60,
        "numRetries": 50,
        "numNoDelayRetries": 3,
        "numMinDelayRetries": 2,
        "numMaxDelayRetries": 35,
        "backoffFunction": "exponential"
    },
    "throttlePolicy": {
        "maxReceivesPerSecond": 10
    },
    "requestPolicy": {
        "headerContentType": "application/json"
    }
}

You set the DeliveryPolicy attribute by calling either the Subscribe or the SetSubscriptionAttributes API action:

Alternatively, you can use AWS CloudFormation for setting this policy as well.

score -1 · Answer 3 · edited May 23 '17 at 12:22

-1

Maybe try this https://stackoverflow.com/a/14664355

before_filter :set_default_response_format

private

def set_default_response_format
  request.format = :json
end

edited May 23 '17 at 12:22

Community

1
1

answered May 02 '15 at 13:25

Hugh

1

This suggestion doesn't address the question/won't work. The problem is with the incoming Request's `Content-Type`. In Rails `request.format` is about the outgoing Response's content type. – stevenharman Oct 04 '17 at 16:42

How to parse request body from Amazon SNS Notification in rails

3 Answers3