aws cli command to subscribe to a topic with filters

Question

I'm trying to write a cross account aws cli command to subscribe to a topic and create a filter for that subscription at the same time. Below is how my command looks like.

aws sns subscribe --topic-arn arn:aws:sns:region:accountId:my_topic --protocol sqs --notification-endpoint arn:aws:sqs:region:differentAccountId:my_sqs_queue --attributes "{'RawMessageDelivery': 'true', 'FilterPolicy': '{\"filter\": [\"value1\", \"value2\"]}'}"

I'm getting below error when I run this.

Unknown options: --attributes, [\value1\,, \value2\]}'}, {'RawMessageDelivery': 'true', 'FilterPolicy': '{" filter\:

I've access to admin access both the aws accounts. Any suggestions on what I'm doing wrong?

EDIT: I'm running this in VS Code powershell terminal in windows.

The problem appears to be related to how the AWS CLI is interpreting the backslashes in the command. Are running that command on Windows or Linux? Instead of using JSON formatting, you could try using the "shorthand syntax" of `KeyName1=string,KeyName2=string`. — John Rotenstein, Apr 16 '20 at 23:50
@JohnRotenstein: Thank you for your time to answer. I'm using powershell in Windows to run this command. I updated the command to use shorthand syntax like `--attributes "RawMessageDelivery=true"` and still getting the same error. I tried `--attributes "RawMessageDelivery=\"true\""` and `--attributes "RawMessageDelivery='true'"` but it threw a similar error `Unknown options: --attributes, RawMessageDelivery=true` — user007, Apr 17 '20 at 18:07

score 20 · Accepted Answer · edited Sep 11 '20 at 02:21

20

There's probably an easier way to do it (eg using --cli-input-json and providing JSON in a file), but I got this working:

aws sns subscribe \
  --topic-arn arn:aws:sns:region:accountId:my_topic \
  --protocol sqs \
  --notification-endpoint arn:aws:sqs:region:differentAccountId:my_sqs_queue \
  --attributes '{\"RawMessageDelivery\": \"true\", \"FilterPolicy\": \"{\\\"filter\\\": [\\\"value1\\\", \\\"value2\\\"]}\"}'

The problem was the JSON included in a string, which needed \" to be escaped as \\\".

edited Sep 11 '20 at 02:21

Phil

157,677
23
242
245

answered Apr 18 '20 at 00:10

John Rotenstein

241,921
22
380
470

3

Thanks for this. The documentation is woeful. I didn't need quite so many escapes in mine... `--attributes '{"FilterPolicy":"{\"filter\":[\"value1\",\"value2\"]}"}'` – Phil Sep 11 '20 at 02:33
I still get the error even with `--attributes '{\"RawMessageDelivery\": \"true\"}'` – Snow May 05 '23 at 11:00

score 4 · Answer 2 · answered Jul 13 '20 at 11:08

This Github repo has an example: https://github.com/Haple/sns-sqs-subscribe

#!/bin/sh

# SETUP

queue_arn=$(awslocal sqs create-queue --queue-name test_queue --output text)

echo "Queue ARN: $queue_arn"

topic_arn=$(awslocal sns create-topic --name test_topic --output text)

echo "Topic ARN: $topic_arn"

subscription_arn=$(awslocal sns subscribe \
    --topic-arn "$topic_arn" \
    --protocol sqs \
    --notification-endpoint "$queue_arn" \
    --output text)

echo "Subscription ARN: $subscription_arn" 

awslocal sns set-subscription-attributes \
    --subscription-arn "$subscription_arn" \
    --attribute-name FilterPolicy \
    --attribute-value "{ \"EVENT_TYPE\": [\"SUCCESS\"] }"

# TEST

awslocal sns publish \
    --topic-arn "$topic_arn" \
    --message "SUCCESS PAYLOAD (SHOULD GO TO THE QUEUE)" \
    --message-attributes '{"EVENT_TYPE" : { "DataType":"String", "StringValue":"SUCCESS"}}'

awslocal sns publish \
    --topic-arn "$topic_arn" \
    --message "ERROR PAYLOAD (SHOULD NOT GO TO THE QUEUE)" \
    --message-attributes '{"EVENT_TYPE" : { "DataType":"String", "StringValue":"ERROR"}}'


awslocal sqs get-queue-attributes \
    --queue-url http://localhost:4576/queue/test_queue \
    --attribute-names All

This one is more concise in my opinion. – Seto Apr 05 '23 at 08:46 — Seto, Apr 05 '23 at 08:46

score 0 · Answer 3 · answered Jan 31 '22 at 15:09

The overall process in the previous answers is OK, but there is a huge issue:

I do not know why no one is mentioning this, but the whole thing doesn't work if you do not set the SQS queue policy document correctly.

The github example doesn't do it and for some reason, AWS documentation doesn't even mention it.

Refer to this question: Set SQS policy document with AWS CLI

I wasted a couple of days troubleshooting this. So hope it helps.

aws cli command to subscribe to a topic with filters

3 Answers3