SNS topic not publishing to SQS

Question

I am trying to prototype a distributed application using SNS and SQS.I have this topic:

arn:aws:sns:us-east-1:574008783416:us-east-1-live-auction

and this queue:

arn:aws:sqs:us-east-1:574008783416:queue4

I created the queue using the JS Scratchpad. I added the subscription using the Console. I AddPermission to the queue using the scratchpad. The queue policy is now:

{  
   "Version":"2008-10-17",
   "Id":"arn:aws:sqs:us-east-1:574008783416:queue4/SQSDefaultPolicy",
   "Statement":[  
      {  
         "Sid":"RootPerms",
         "Effect":"Allow",
         "Principal":{  
            "AWS":"574008783416"
         },
         "Action":"SQS:*",
         "Resource":"arn:aws:sqs:us-east-1:574008783416:queue4"
      }
   ]
}

I have an email subscription on the same topic and the emails arrive fine but the messages never arrive on the queue. I've tried SendMessage directly to the queue - rather than via SNS - using Scratchpad and it works fine. Any ideas why it won't send to the queue?

score 39 · Accepted Answer · edited May 19 '15 at 11:47

39

This was posted a while back on the AWS forums: https://forums.aws.amazon.com/thread.jspa?messageID=202798

Then I gave the SNS topic the permission to send messages to the SQS queue. The trick here is to allow all principals. SNS doesn't send from your account ID -- it has its own account ID that it sends from.

edited May 19 '15 at 11:47

Christopher Hackett

6,042
2
31
41

answered May 09 '11 at 02:39

Skyler Johnson

3,833
1
20
6

Thank you, thank you. Had an open ticket with AWS support and have begged them to add this answer to the Policy Examples in the docs. – user427875 May 11 '11 at 00:05
3

You'll want to restrict access to the sourceArn: https://docs.aws.amazon.com/sns/latest/dg/sns-access-policy-use-cases.html#sns-publish-messages-to-sqs-queue – Mmm Donuts Mar 06 '20 at 14:34
1

This is very bad answer. By doing that everyone can send messages from this topic. Users should seek policies that are allowing them to do things they desire while being the most restrictive policies they can have - it's safer. i think that your SQS is going to be public after using * as principal. – Shoter Jul 31 '20 at 09:27
example: https://docs.amazonaws.cn/en_us/sns/latest/dg/subscribe-sqs-queue-to-sns-topic.html#SendMessageToSQS.sqs.permissions – borjab Apr 16 '21 at 10:10
1

The link to the aws forum has been archived, it is always better option to put the context of the answer you are proposing along with the link. – Shadman R Jun 09 '22 at 03:34

score 27 · Answer 2 · answered Feb 08 '19 at 18:39

27

Adding to Skyler's answer, if like me you cringe at the idea of allowing any principal (Principal: '*'), you can restrict the principal to SNS:

Principal:
  Service: sns.amazonaws.com

Although this behavior is undocumented, it works.

answered Feb 08 '19 at 18:39

spg

9,309
4
36
41

1

This is correct answer to the problem. By doing things this way you will not expose your SQS to whole world. – Shoter Jul 31 '20 at 15:37
Wow this was the working solution for me after hours of trying out others, Thanks a ton :) – Sandeep K Nair Mar 01 '21 at 10:50
1

I found this documented on step 2 of this guide: https://docs.aws.amazon.com/sns/latest/dg/subscribe-sqs-queue-to-sns-topic.html – dnc253 Jun 03 '22 at 19:57

score 23 · Answer 3 · answered Jul 31 '20 at 14:06

Most of the answers (beside @spg answer) propose usage of principal: * - this is very dangerous practice and it will expose your SQS to whole world.

From AWS docs

For resource-based policies, such as Amazon S3 bucket policies, a wildcard (*) in the principal element specifies all users or public access.
We strongly recommend that you do not use a wildcard in the Principal element in a role's trust policy unless you otherwise restrict access through a Condition element in the policy. Otherwise, any IAM user in any account in your partition can access the role.

Therefore it is strongly not recommended to use this principal.

Instead you need to specify sns service as your principal:

"Principal": {
        "Service": "sns.amazonaws.com"
},

Example policy:

{
  "Version": "2012-10-17",
  "Id": "Policy1596186813341",
  "Statement": [
    {
      "Sid": "Stmt1596186812579",
      "Effect": "Allow",
      "Principal": {
        "Service": "sns.amazonaws.com"
      },
      "Action": [
        "sqs:SendMessage",
        "sqs:SendMessageBatch"
      ],
      "Resource": "Your-SQS-Arn"
    }
  ]
}

With this policy sns will be able to send messages to your SQSs.

There are more permissions for SQS but from what I see SendMessage and SendMessageBatch should be enough for SNS->SQS subscribtion.

This is a fine answer. I'm surprised when I connect SNS to SQS in AWS console, it doesn't do the permission part automatically. — piggybox, Jan 06 '22 at 17:49
I like the idea of the extra security, but I have a problem adding this policy to the SNS topic Access Policy. The problem being reported saying: "Policy statement action out of service scope!" After reading the topic Access Policy info - it makes sense. Topic Access Policy controls who can access the topic - not the queue. Trying the same with the queue... — tishma, Feb 09 '23 at 12:39

kichik · Answer 4 · 2020-07-31T17:41:59.347

Here's a full CloudFormation example of Skyler's answer

{
  "Resources": {
    "MyTopic": {
      "Type": "AWS::SNS::Topic"
    },
    "MyQueue": {
      "Type": "AWS::SQS::Queue"
    },
    "Subscription": {
      "Type" : "AWS::SNS::Subscription",
      "Properties" : {
          "Protocol" : "sqs",
          "TopicArn" : {"Ref": "MyTopic"},
          "Endpoint": {"Fn::GetAtt": ["MyQueue", "Arn"]}
        }
    },
    "QueuePolicy": {
      "Type": "AWS::SQS::QueuePolicy",
      "Properties": {
        "Queues": [
          {"Ref": "MyQueue"}
        ],
        "PolicyDocument": {
          "Version": "2012-10-17",
          "Statement": [
            {
              "Sid": "allow-sns-messages",
              "Effect": "Allow",
              "Principal": {"Service": "sns.amazonaws.com"},
              "Action": "sqs:SendMessage",
              "Resource": {"Fn::GetAtt": ["MyQueue", "Arn"]},
              "Condition": {
                "ArnEquals": {
                  "aws:SourceArn": {"Ref": "MyTopic"}
                }
              }
            }
          ]
        }
      }
    }
  }
}

Amazon has more options in their Sending Amazon SNS Messages to Amazon SQS Queues document.

As far as I can tell, the Statement needs to include a Resource property: "Resource": { "Fn::GetAtt": ["MyQueue", "Arn" ] }, — Digitalkapitaen, Nov 12 '18 at 16:04

score 13 · Answer 5 · answered Mar 22 '22 at 03:18

I just experienced this and took me a while to figure out why:

If I create a SQS subscription from the SNS console, it does not add necessary permissions to the SQS access policy.

If I create the subscription in the SQS console to the same SNS, it does.

Ulad Kasach · Answer 6 · 2021-05-24T14:39:08.940

Like the other answers mentioned, you must opt in and grant permission to this SNS topic to publish to your SQS queue.

If you use terraform, you can use the sqs_queue_policy resource.

Here is an example:

resource "aws_sqs_queue_policy" "your_queue_policy" {
    queue_url = "${aws_sqs_queue.your_queue.id}"

    policy = <<POLICY
{
  "Version": "2012-10-17",
  "Id": "sqspolicy",
  "Statement": [
    {
      "Sid": "First",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "sqs:SendMessage",
      "Resource": "${aws_sqs_queue.your_queue.arn}",
      "Condition": {
        "ArnEquals": {
          "aws:SourceArn": "${aws_sns_topic.your_topic.arn}"
        }
      }
    }
  ]
}
POLICY
}

jackotonye · Answer 7 · 2020-08-05T01:15:54.450

Old question but using an AWS SDK version > 1.10

Check out the docs SQS-SNS sendMessage Permission

private static void updateQueuePolicy(AmazonSQS sqs, String queueURL, String topicARN) {
   Map<String, String> attributes = new HashMap<String, String>(1);
   Action actions = new Action() {
       @Override
       public String getActionName() {
           return "sqs:SendMessage"; // Action name
       }
   };
   Statement mainQueueStatements = new Statement(Statement.Effect.Allow)
           .withActions(actions)
           .withPrincipals(new Principal("Service", "sns.amazonaws.com"))
           .withConditions(
               new Condition()
               .withType("ArnEquals")
               .withConditionKey("aws:SourceArn")
               .withValues(topicARN)
           );

   final Policy mainQueuePolicy = new Policy()
        .withId("MainQueuePolicy")
        .withStatements(mainQueueStatements);

   attributes.put("Policy", mainQueuePolicy.toJson());

   updateQueueAttributes(sqs, queueURL, attributes);
}

Which outputs a policy similar to

{
   Policy={
      "Version":"2012-10-17",
       "Id":"MainQueuePolicy",
       "Statement":
            [
              {
                 "Sid":"1",
                 "Effect":"Allow",
                 "Principal": {
                   "Service": "sns.amazonaws.com"
                 },
                 "Action":["sqs:SendMessage"],
                 "Condition":
                   {"ArnEquals":
                     {"aws:SourceArn":["arn:aws:sns:us-east-1:3232:testSubscription"]}
                   }
              }
            ]
      }
 }

Thanks for updating your answer to faciliate correct Principal! :) — Shoter, Apr 27 '21 at 09:24

score 2 · Answer 8 · answered Jun 17 '21 at 13:17

2

If you have enabled encryption on your queue, can also be a reason for SNS not being able to put message on subscriber queue. You need to give access to SNS to that KMS key.

This article explains how to solve this problem:

answered Jun 17 '21 at 13:17

horizon7

1,113
14
18

score 1 · Answer 9 · edited Jul 14 '20 at 18:24

1

only you need subscribe the queue to the topic from the queue console.

step one: select the queue
step two: queue Actions
step three: Subscribe Queue to SNS Topic
step choose: the topic
end.

edited Jul 14 '20 at 18:24

Sebastián Palma

32,692
6
40
59

answered May 25 '20 at 00:30

Natanael Rocha

19
2

score 0 · Answer 10 · answered Sep 08 '22 at 02:34

It's by AWS::SQS::QueuePolicy.

You need define a this kind of policy to allow specific SNS perform actions to specific SQS.

QueuePolicy:
  Type: AWS::SQS::QueuePolicy
  Properties:
    Queues:
    - Ref: MyQueue1
    - Fn::Sub: arn:aws:sqs:us-east-1:${AWS::AccountId}:my-queue-*
    PolicyDocument:
      Version: '2012-10-17'
      Statement:
      - Sid: allow-sns-messages
        Effect: Allow
        Principal:
          Service: sns.amazonaws.com
        Action: sqs:SendMessage
        Resource:
          - Fn::Sub: arn:aws:sqs:us-east-1:${AWS::AccountId}:my-queue-*
        Condition:
          ArnEquals:
            aws:SourceArn:
              - Fn::Sub: arn:aws:sns:us-east-1:${AWS::AccountId}:source-sns-*

With Lambda this kind of policies dont apply. Please if anyone know why please share it.

SNS topic not publishing to SQS

10 Answers10