bash string formatting within a quote

Question

I’m trying to generalize a command inside a bash script but I’m stuck with some string formatting. The code I’m trying to recreate (which works)

curl -X POST -H 'Content-Type: application/json' $CLUSTER -d '{
      "source" : "s3://blah/part-00004-9d2ba62f-496e-4cfd-9001-f40f0e33e927-c000.csv",
      "format" : "csv"
    }'

with the below command (which doesn’t work)

filename='part-00004-9d2ba62f-496e-4cfd-9001-f40f0e33e927-c000.csv'
curl -X POST -H 'Content-Type: application/json' $CLUSTER -d '{
    "source" : "s3://blah/$filename",
    "format" : "csv"
    }'

I also tried out the tip from Expansion of variables inside single quotes in a command in Bash but it didn’t pan out.

"source" : '"s3://blah/$filename"',

Any ideas?

You don't get variable inside single quotes. Either use double quotes outside and escape the literal double quotes which you need, i.e. `"\"s3: ....`, or put the string together piecewise, avoiding the need to escape: `'"s3:'"//blah/$filename"'"'`. — user1934428, Jul 10 '20 at 07:48

score 2 · Accepted Answer · answered Jul 10 '20 at 07:24

Try this (see the "'"):

filename='part-00004-9d2ba62f-496e-4cfd-9001-f40f0e33e927-c000.csv'
curl -X POST -H 'Content-Type: application/json' $CLUSTER -d '{
    "source" : "'"s3://blah/$filename"'",
    "format" : "csv"
    }'

The first "'" is "double quote symbol for the -d argument, switch single quote escaping off, start double quote escaping". The second "'" is similar.

score 1 · Answer 2 · answered Jul 10 '20 at 07:33

I'm a fan of using jq to build JSON from user-supplied values in shell scripts, as that way if your filename (Or whatever) happens to have any characters in it that need special treatment in JSON, it'll be handled automatically. Something like

filename='part-00004-9d2ba62f-496e-4cfd-9001-f40f0e33e927-c000.csv'
curl -X POST \
  -H 'Content-Type: application/json' \
  "$CLUSTER" \
  -d "$(jq -n --arg filename "$filename" '{source:"s3://blah/\($filename)",format:"csv"}')"

bash string formatting within a quote

2 Answers2