bash function with third arg is an array

Question

I'm trying to write a bash function that does the following:

If there is no 3rd argument, run a command.
If there is a third argument, take every argument from the third one and run a command.

The problem I have is the last bit of the command --capabilities CAPABILITY_IAM in the else statement that I don't want to pass in all the time if I have multiple parameters.

An error occurred (InsufficientCapabilitiesException) when calling the CreateStack operation: Requires capabilities : [CAPABILITY_NAMED_IAM]
// that means I need to pass in --capabilities CAPABILITY_IAM

Is there a way to tell bash that: hey, take all the args from the 3rd one, then add the --capabilities CAPABILITY_IAM after? Like in JavaScript I can do this:

function allTogetherNow(a, b, ...c) {
  console.log(`${a}, ${b}, ${c}. Can I have a little more?`);
}

allTogetherNow('one', 'two', 'three', 'four')

Here's my function:

cloudformation_create() {
    if [ -z "$3" ]; then
        aws cloudformation create-stack --stack-name "$1" --template-body file://"$2" --capabilities CAPABILITY_IAM
    else
        aws cloudformation create-stack --stack-name "$1" --template-body file://"$2" --parameters "${@:3}" --capabilities CAPABILITY_IAM
    fi
}

And the 3rd and so on parameters look like this if I don't use a bash function:

aws cloudformation create-stack --stack-name MY_STACK_NAME --template-body file://MY_FILE_NAME --parameters ParameterKey=KeyPairName,ParameterValue=TestKey ParameterKey=SubnetIDs,ParameterValue=SubnetID1 --capabilities CAPABILITY_IAM

Update 22 May 2019:

Following Dennis Williamson's answer below. I've tried:

Passing the parameters in the AWS way:

cloudformation_create STACK_NAME FILE_NAME ParameterKey=KeyPairName,ParameterValue=TestKey ParameterKey=SubnetIDs,ParameterValue=SubnetID1

Got error:

An error occurred (ValidationError) when calling the CreateStack operation: Parameters: [...] must have values

Pass in as a string:

cloudformation_create STACK_NAME FILE_NAME "ParameterKey=KeyPairName,ParameterValue=TestKey ParameterKey=SubnetIDs,ParameterValue=SubnetID1"

Got error:

An error occurred (ValidationError) when calling the CreateStack operation: ParameterValue for ... is required

Pass in without ParameterKey and ParameterValue:

cloudformation_create STACK_NAME FILE_NAME KeyPairName=TestKey SubnetIDs=SubnetID1

Got error:

Parameter validation failed:
Unknown parameter in Parameters[0]: "PARAM_NAME", must be one of: ParameterKey, ParameterValue, UsePreviousValue, ResolvedValue
// list of all the params with the above error

Pass in without ParameterKey and ParameterValue and as a string. Got error:

arameter validation failed:
Unknown parameter in Parameters[0]: "PARAM_NAME", must be one of: ParameterKey, ParameterValue, UsePreviousValue, ResolvedValue

I tried Alex Harvey's answer and got this:

An error occurred (ValidationError) when calling the CreateStack operation: Template format error: unsupported structure.

@DennisWilliamson it needs the `--capabilities CAPABILITY_IAM` at the end. For some reason, my bash function takes all the args but doesn't add the capability at the end. — Viet, May 21 '19 at 22:56
I don't think it's your function that's consuming that last option and its argument. I think it may be the malformed argument to `--parameters` that's causing that argument to consume the other. See my answer. — Dennis Williamson, May 21 '19 at 23:05
I changed to comma and got this error: ```bash Parameter validation failed: Unknown parameter in Parameters[0]: "PARAM_NAME", must be one of: ParameterKey, ParameterValue, UsePreviousValue, ResolvedValue ``` — Viet, May 22 '19 at 14:35
What happens when you use your Bash function (just below "Here's my function:" above) which uses "${@:3}" (instead of the asterisk)? — Dennis Williamson, May 22 '19 at 14:37
I got that error but I've realised my mistake now. Thank you. @DennisWilliamson. — Viet, May 22 '19 at 14:40
We don't add "solved" to question titles on Stack Overflow. Please post your solution as an answer. You can even mark your own answer as accepted. — Dennis Williamson, May 22 '19 at 15:10
@DennisWilliamson thank you. I was trying to save you (and others) time because I can only mark my own answer as "correct answer" tomorrow. — Viet, May 22 '19 at 15:14
@Viet, fyi, the "unsupported structure" is because of a typo on my part ; I missed the "file://" bit. I've updated. — Alex Harvey, May 22 '19 at 15:29

score 3 · Answer 1 · answered May 21 '19 at 23:03

Based on LeBlue's answer and a quick read of the docs, it looks like you need to build the argument to --parameters from the arguments to the function:

cloudformation_create () {
local IFS=,
local parameters="${*:3}"

#if block not shown
aws cloud_formation ... --parameters "$parameters" ...

this presumes that your function is called like this:

cloudformation_create foo bar baz=123 qux=456

that is, with the key, value pairs already formed.

The snippet above works by setting IFS to a comma. Using $* inside quotes causes the elements contained in it to be joined using the first character of IFS. If you need to make use of the word splitting features in another part of your function, you may want to save the current value of IFS before changing it then restore it after the joining assignment.

As a result, the expansion of $parameters will be baz=123,qux=456

I'm sorry. This doesn't work. I'll update my question with what I got from your code. — Viet, May 22 '19 at 13:35

score 2 · Answer 2 · answered May 21 '19 at 21:33

2

I am not sure that answers your question but maybe it will help you.

$# is the number of parameters.

$@ will include all parameters and you can pass that.

#!/bin/bash

foo()
{
    echo "params in foo: " $#
    echo "p1: " $1
    echo "p2: " $2
    echo "p3: " $3
}

echo "number of paramters: " $#

foo $@ 3   # pass params and add one to the end

Call:

./test.sh 1 2

Output:

number of paramters:  2
params in foo:  3
p1:  1
p2:  2
p3:  3

answered May 21 '19 at 21:33

chris01

10,921
9
54
93

Thank you @Chris. Unfortunately, this is not what I'm looking for, I need something like this: `foo $1 $2 $3 $4 (and so on) then does: do $1 and $2 and do $3 $4 $5 then do the last bit` – Viet May 21 '19 at 21:37

score 1 · Answer 3 · answered May 21 '19 at 22:34

I suspect the parameter expansion is wrong, as --parameters probably needs to have one argument. Either quote all arguments to cloudformation_create that need to end up as value for the --parameters flag:

cloudformation_create "the-stack" "the-filename" "all the parameters"

or rewrite the function to not expand into multiple arguments with "$*" (merge every arg into one)

cloudformation_create () {
    ...
    else
        aws cloudformation ... --parameters "${*:3}" --capabilities CAPABILITY_IAM
    fi
}

This will preserve all values as one string/argument, both will translate to:

aws cloudformation  ... --parameters "all other parameters" --capabilities CAPABILITY_IAM

as opposed to your version:

aws cloudformation ... --parameters "all" "other" "parameters" --capabilities CAPABILITY_IAM

I think you may be close. The [docs](https://docs.aws.amazon.com/cli/latest/reference/cloudformation/create-stack.html) indicate that the argument to `--parameters` is a comma-delimited list of `key=value` pairs. — Dennis Williamson, May 21 '19 at 22:48
I think you're close. Like @DennisWilliamson wrote above, the `--parameters` is a list. I've updated my question above. — Viet, May 21 '19 at 22:54

Alex Harvey · Answer 4 · 2019-05-22T15:29:50.870

0

I would write it this way:

cloudformation_create() {
  local stack_name=$1
  local template_body=$2
  shift 2 ; local parameters="$@"

  local command=(aws cloudformation create-stack 
    --stack-name "$stack_name"
    --template-body "file://$template_body")

  [ ! -z "$parameters" ] && \
    command+=(--parameters "$parameters")

  command+=(--capabilities CAPABILITY_IAM)

  ${command[@]}
}

Note:

calling shift 2 results in $3 being rotated to $1 such that you can just use $@ as normal.

edited May 22 '19 at 15:29

answered May 22 '19 at 12:12

Alex Harvey

14,494
5
61
97

You can do `shift 2`. Putting commands, options and arguments in variables is [discouraged](https://mywiki.wooledge.org/BashFAQ/050). – Dennis Williamson May 22 '19 at 12:28

score 0 · Accepted Answer · answered May 22 '19 at 14:45

First of all, thank you all for your help.

I've realised the issue (and my mistake):

AWS returned the error with Requires capabilities : [CAPABILITY_NAMED_IAM] and my function has [CAPABILITY_IAM]. Depends on the template with params related to creating IAM, [CAPABILITY_NAMED_IAM] or [CAPABILITY_IAM] is required. I found the answer here helpful.

So in my case, the bash function is good, for the template I was trying to create, I need to pass in --capabilities CAPABILITY_NAMED_IAM. I've tried it and it works.

bash function with third arg is an array

5 Answers5