How do you add spaces for aws cloudformation deploy --parameter-overrides and/or --tags?

Question

I am trying to get spaces into the tags parameter for the aws cli and it works if I hardcode it but not if I use bash variables. What is going on and how do I fix it?

This works with out spaces:

aws cloudformation deploy  \
      --template-file /path_to_template/template.json \ 
      --stack-name my-new-stack \
      --tags Key1=Value1 Key2=Value2

This works with out spaces but with variables:

tags="Key1=Value1 Key2=Value2" 
aws cloudformation deploy \
  --template-file /path_to_template/template.json \ 
  --stack-name my-new-stack \
  --tags $tags

This works with spaces:

aws cloudformation deploy  \
  --template-file /path_to_template/template.json \ 
  --stack-name my-new-stack \
  --tags 'Key1=Value1' 'Key Two=Value2'

This does not work, spaces and variables:

tags="'Key1=Value1' 'Key Two=Value2'"

aws cloudformation deploy  \
  --template-file /path_to_template/template.json \ 
  --stack-name my-new-stack \
  --tags $tags

Attempting to fix bash expansion, also does not work, spaces and variables:

tags="'Key1=Value1' 'Key Two=Value2'"

aws cloudformation deploy  \
  --template-file /path_to_template/template.json \ 
  --stack-name my-new-stack \
  --tags "$tags"

Attempting to fix bash expansion, also does not work, spaces and variables:

tags="'Key1=Value1' 'Key Two=Value2'"

aws cloudformation deploy  \
  --template-file /path_to_template/template.json \ 
  --stack-name my-new-stack \
  --tags "$(printf '%q' $tags)"

Error:

Invalid parameter: Tags Reason: The given tag(s) contain invalid characters (Service: AmazonSNS; Status Code: 400; Error Code: InvalidParameter; Request ID

score 2 · Answer 1 · answered Dec 19 '19 at 00:14

2

Would you please try:

tags=('Key1=Value1' 'Key Two=Value2')

aws cloudformation deploy  \
  --template-file /path_to_template/template.json \ 
  --stack-name my-new-stack \
  --tags "${tags[@]}"

answered Dec 19 '19 at 00:14

tshiono

21,248
2
14
22

Does not work for `create-stack` (instead of `deploy`), returns error: Parameter validation failed: Missing required parameter in Tags[0]: "Key" Missing required parameter in Tags[0]: "Value" Unknown parameter in Tags[0]: "Key1", must be one of: Key, Value – Putnik May 05 '23 at 10:16

score 1 · Accepted Answer · answered Dec 20 '19 at 19:20

Stealing some ideas from https://github.com/aws/aws-cli/issues/3274 I was able to get this working by doing the following

    deploy=(aws cloudformation deploy 
            ...
            --tags $(cat tags.json | jq '.[] | (.Key + "=" + .Value)'))

    eval $(echo ${deploy[@]})

With a tags.json file structure of

[
    {
        "Key": "Name With Spaces",
        "Value": "Value With Spaces"
    },
    {
        "Key": "Foo",
        "Value": "Bar"
    }
]

Gilles Quénot · Answer 3 · 2019-12-18T21:35:32.893

Try this :

tags="'Key1=Value1' 'Key Two=Value2'"

aws cloudformation deploy  \
  --template-file /path_to_template/template.json \ 
  --stack-name my-new-stack \
  --tags "$tags"
#        ^     ^
#     double quotes

Learn how to quote properly in shell, it's very important :

"Double quote" every literal that contains spaces/metacharacters and every expansion: "$var", "$(command "$var")", "${array[@]}", "a & b". Use 'single quotes' for code or literal $'s: 'Costs $5 US', ssh host 'echo "$HOSTNAME"'. See
http://mywiki.wooledge.org/Quotes
http://mywiki.wooledge.org/Arguments
http://wiki.bash-hackers.org/syntax/words

vfrank66 · Answer 4 · 2022-02-11T12:58:18.223

As of 2022-02 this was still an issue described

here

here also

and a little here

@esolomon is correct you have to array expansion. His answer which works just fine:

deploy=(aws cloudformation deploy 
            ...
            --tags $(cat tags.json | jq '.[] | (.Key + "=" + .Value)'))

eval $(echo ${deploy[@]})

The actual problem is a result of the shell environment (bin/bash here) that is used in combination with how python cli executable's handling of values. Since the aws cloudformation deploy does not standardize the input but expects the shell program to standardize array input this was causing my problem.

So my errors with the --debug flag turned on produced the first response which is the error and the second response is the expected input into aws cloudformation deploy

Error input:

2022-02-10 17:32:28,137 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['cloudformation', 'deploy', '--region', 'us-east-1', ..., '--parameter-overrides', 'PARAM1=dev PARAM2=blah', '--tags', "TAG1='Test Project' TAG2='blah'...", '--debug']

Expected input:

2022-02-10 17:39:40,390 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['cloudformation', 'deploy', '--region', 'us-east-1', ..., '--parameter-overrides', 'PARAM1=dev', 'PARAM2=blah', '--tags', "TAG2='Test Project'", 'TAG2=blah',..., '--debug']

I was unexpectedly sending in a string instead of array of strings this error resulted in several errors depending on how I sent it: example TAG: TAG1=Test Project

['Project'] value passed to --tags must be of format Key=Value

the means IFS needs to be set to something other than the default ' \t\n', solution below

An error occurred (ValidationError) when calling the CreateChangeSet operation: 1 validation error detected: Value 'Test Project Tag2=Value2 ...' at 'tags.1.member.value' failed to satisfy constraint: Member must have length less than or equal to 256

the error starts after the first = this error means that I am sending in one long string instead of array items, as seen here when doing [*] instead of [@] aws cloudformation deploy ... --tags "${TAGS[*]}" diff between [*] and [@]

To fix this the most important thing was that IFS needed to be set to anything other than ' \t\n' and secondly I still need to do array expansion with [@] and could not input a string. The --parameter-overrides for me did not have this problem even though similar variable loading BECAUSE it did not have a string.

This was my solution, my params and tags input is all over the place, spaces + sometimes arrays + bad indenting thus the sed:


export IFS=$'\n'
# Build up the parameters and Tags
PARAMS=($(jq '.[] | .ParameterKey + "=" + if .ParameterValue|type=="array" then .ParameterValue | join(",") else .ParameterValue end ' parameters-${environment}.json \
    | sed -e 's/"//g' \
    | sed -e $'s/\r//g' | tr '\n' ' '))
TAGS=("$(jq -r '.[] | [.Key, .Value] | "\(.[0])=\(.[1])"'  tags-common.json)")
TAGS=($TAGS "$(jq -r '.[] | [.Key, .Value] | "\(.[0])=\(.[1])"' tags-${environment}.json)")

aws cloudformation deploy \
    --region "${REGION}" \
    --no-fail-on-empty-changeset \
    --template-file "stack-name-cfn-transform.yaml" \
    --stack-name "stack-name-${environment}" \
    --capabilities CAPABILITY_NAMED_IAM \
    --parameter-overrides "${params[@]}" \
    --tags "${TAGS[@]}" \
    --profile "${PROFILE}"

parameters file

[
    {
        "ParameterKey": "Environment",
        "ParameterValue": "dev"
    }
]

tags file - both common and environment specific tag files have same format

[
    {
        "Key": "TAG1",
        "Value": "Test Project"
    },
    {
        "Key": "Name With Spaces",
        "Value": "Value With Spaces"
    },
    {
        "Key": "Foo",
        "Value": "Bar"
    }
]

score 0 · Answer 5 · answered Mar 11 '22 at 22:11

I resolved this scenario using options below:

"scripts": { "invoke": "sam ... --parameter-overrides \"$(jq -j 'to_entries[] | \"\\(.key)='\\\\\\\"'\\(.value)'\\\\\\\"''\\ '\"' params.json)\"" }

Or

sam ... --parameter-overrides "$(jq -j 'to_entries[] | "\(.key)='\\\"'\(.value)'\\\"''\ '"' params.json)"

How do you add spaces for aws cloudformation deploy --parameter-overrides and/or --tags?

5 Answers5