8

I am trying to set redrive policy for SQS using the AWS CLI Command below , but seeing an error related to redrive JSON. Can you please let me know how I can fix this?

redrive_policy="{\"RedrivePolicy\":{\"deadLetterTargetArn\":\"$dlq_arn\",\"maxReceiveCount\":\"15\"}}"

AWS CLI COMMAND

aws sqs set-queue-attributes --queue-url https://queue.amazonaws.com/12345678/test-queue --attributes $redrive_policy --region=us-east-1

Error Message

Parameter validation failed: Invalid type for parameter Attributes.RedrivePolicy, value: OrderedDict([(u'deadLetterTargetArn', u'arn:aws:sqs:us-east-1:12345678:dlq'), (u'maxReceiveCount', u'15')]), type: , valid types:

Punter Vicky
  • 15,954
  • 56
  • 188
  • 315

3 Answers3

11

Have you tried just creating the JSON in a separate file and passing it as an argument to your AWS CLI command? I find it's difficult to get all of the escaping correct when passing the JSON as a parameter. So you'd basically do it as the example shows in the AWS documentation:

https://docs.aws.amazon.com/cli/latest/reference/sqs/set-queue-attributes.html#examples

  1. So first you'd create a new file called "set-queue-attributes.json" like so:
    {
      "DelaySeconds": "10",
      "MaximumMessageSize": "131072",
      "MessageRetentionPeriod": "259200",
      "ReceiveMessageWaitTimeSeconds": "20",
      "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:us-east-1:80398EXAMPLE:MyDeadLetterQueue\",\"maxReceiveCount\":\"1000\"}",
      "VisibilityTimeout": "60"
    }
  1. Then run the command like this:
aws sqs set-queue-attributes --queue-url https://sqs.us-east-1.amazonaws.com/80398EXAMPLE/MyNewQueue --attributes file://set-queue-attributes.json --region=us-east-1
Brod
  • 1,377
  • 12
  • 14
  • 1
    Thanks , I just figured that I missed the double quotes for redrive policy. It worked fine after I added that – Punter Vicky Jul 30 '19 at 16:06
  • Any tips on how to remove it afterwards? I tried omitting the `RedrivePolicy` field from the JSON but no luck so far... – Ondrej Burkert Dec 21 '20 at 16:48
  • @PunterVicky - I am trying to do the exact thing with the Policy attribute. Getting an identical error. What did you fix? – JDBennett Aug 10 '21 at 15:23
  • @JDBennett it’s been quite a while but I believe I removed the double quotes when assigning the value to redrive_policy variable – Punter Vicky Aug 11 '21 at 16:11
  • I got it figured out. It is the double quotes - need to use single quotes wrapping escaped json. Double quotes remove the escaping and turn it back into proper json. – JDBennett Aug 12 '21 at 12:55
2

if you want to run in the same command you can use this example:

aws sqs set-queue-attributes \
--queue-url https://sqs.us-east-1.amazonaws.com/80398EXAMPLE/MyNewQueue \
--attributes '{
    "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:us-east-1:80398EXAMPLE:MyDeadLetterQueue\",\"maxReceiveCount\":\"1000\"}",
    "MessageRetentionPeriod": "259200",
    "VisibilityTimeout": "90"
}'
Pedro Bacchini
  • 876
  • 10
  • 13
1

Three Methods to achieve this:

Note: The solutions also work on any other AWS CLI commands that require a stringified JSON

1. Using the Command-line JSON processor jq (Recommended)

This method is recommended because of many reasons:

  • I've found jq a handy tool to use when working with AWS CLI as the need to stringify JSON comes up quite frequently.

Install for Ubuntu: sudo apt install jq

Basic Options:

  • jq -R: Returns the stringified JSON
  • jq -c: Eliminates spacing and newline characters

The benefit is that you can write JSON as JSON and Pipe the result into the jq -R command.

Method 1:

aws sqs set-queue-attributes \
    --queue-url "https://sqs.ap-south-1.amazonaws.com/IAMEXAMPLE12345678/ExampleQueue" \
    --attributes RedrivePolicy=$(echo '{"maxReceiveCount":500,"deadLetterTargetArn":"arn:aws:sqs:ap-south-1:IAMEXAMPLE12345678:ExampleDeadLetterQueue"}' | jq -R)

OR if you have a sqs-redrive-policy.json file:

Method 2:

In sqs-redrive-policy.json,

{
  "maxReceiveCount": 500,
  "deadLetterTargetArn": "arn:aws:sqs:ap-south-1:IAMEXAMPLE12345678:ExampleDeadLetterQueue"
}

Run in Command Line:

   aws sqs set-queue-attributes \
    --queue-url "https://sqs.ap-south-1.amazonaws.com/IAMEXAMPLE12345678/ExampleQueue" \ 
    --attributes RedrivePolicy=$(cat ~/path/to/file/sqs-redrive-policy.json | jq -c | jq -R)
  • As you can see the second benefit is that you can isolately modify only the --redrive-policy without having to touch any of the other attributes.

Common Confusion: A confusion is the name set-queue-attributes (it would be better named put-queue-attributes). as it doesn't overwrite all attributes but only overwrites the attributes mentioned with the command. So if you already set a Policy attribute earlier during create-queue, this will not overwrite the Policy to null. In other words, this is safe to use.

2. Using a stringified JSON

This is a pain to be honest, and I avoid this.

aws sqs set-queue-attributes \
--queue-url "https://sqs.us-east-1.amazonaws.com/IAMEXAMPLE12345678/ExampleQueue" \
--attributes '{
    "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:ap-south-1:IAMEXAMPLE12345678:ExampleDeadLetterQueue\",\"maxReceiveCount\":\"500\"}",
}'

3. Use a filePathURL to the JSON file for attributes.json NOT sqs-redrive-policy.json

This is my last preference.

Reason:

  • This means setting all the attributes specified in the attributes.json file again at a single go.
  • Doesn't escape the pain of writing stringified JSON as text.

In attributes.json,

{
  "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:ap-south-1:IAMEXAMPLE12345678:ExampleDeadLetterQueue\", \"maxReceiveCount\":\"5\"}"
}

Run in command line:

aws sqs set-queue-attributes \
    --queue-url "https://sqs.ap-south-1.amazonaws.com/IAMEXAMPLE12345678/ExampleQueue" \
    --attributes file:///home/yourusername/path/to/file/attributes.json
Jayanta Samaddar
  • 101
  • 3