Can't use shell variable in curl data payload

Question

Given is the following shell script

#!/usr/bin/env bash
log=`git log discord-report..development --no-merges --decorate --format=format:'%C(bold green)(%ar)%C(reset) %C(white)%s%C(reset) %C(dim white)- %an'`
echo "$log"

curl -i -H "Content-Type: application/json" -X POST --data "{ \"username\" : \"CoopR Development Info\", \"content\" : \"@everyone\n**Development** Branch has been updated.\nPlease pull the newest state!\n Changelog: '$log'  \" }" https://discordapp.com/api/webhooks/XXX/XXX

echo "$log" yields the following:

(36 minutes ago) chore(demo): add vr development mission - xetra11
(6 hours ago) refactor(tasks): move aar to tasks addon and use more cba notify funcs - xetra11
(2 days ago) refactor(tasks): temp test deactivation - xetra11
(3 days ago) feat(tasks): add report reduncancy check function - xetra11

The whole script output is:

$ ./test.sh
(38 minutes ago) chore(demo): add vr development mission - xetra11
(6 hours ago) refactor(tasks): move aar to tasks addon and use more cba notify funcs - xetra11
(2 days ago) refactor(tasks): temp test deactivation - xetra11
(3 days ago) feat(tasks): add report reduncancy check function - xetra11
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   512  100    58  100   454     66    518 --:--:-- --:--:-- --:--:--   585HTTP/2 400
date: Tue, 19 Mar 2019 22:27:24 GMT
content-type: application/json
content-length: 58
set-cookie: __cfduid=XXX; expires=Wed, 18-Mar-20 22:27:24 GMT; path=/; domain=.discordapp.com; HttpOnly
strict-transport-security: max-age=31536000; includeSubDomains
x-ratelimit-limit: 5
x-ratelimit-remaining: 4
x-ratelimit-reset: 1553034447
via: 1.1 google
alt-svc: clear
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 4ba2dfa01fbcc4a1-DUS

{"code": 50006, "message": "Cannot send an empty message"}

I tried everything so far but I just can not get this to work and have my git log result send via the curl post.

What I want

At the moment curl returns an error. I want the content of $log to be included into the -d option of curl.

--data "{ \"username\" : \"CoopR Development Info\", \"content\" : \"@everyone\n**Development** Branch has been updated.\nPlease pull the newest state!\n Changelog: '$log'  \" }"

This is how I defined it atm (I tried a lot of formats but none worked)

`I tried everything so far but I just can not get this to work and have my git log result send via the curl post.` it's the last comment. Not very visible to be honest — xetra11, Mar 19 '19 at 22:30
Dude if something is not working I surely ask how to get it working. Please. — xetra11, Mar 19 '19 at 22:32
Please define "working". What do you want your bash script to do instead of what it is currently doing? — Code-Apprentice, Mar 19 '19 at 22:39
I want it to not `{"code": 50006, "message": "Cannot send an empty message"}` yield this message and actually add the `$log` variable content into the `-d` option string of `curl` — xetra11, Mar 19 '19 at 22:40
First try replacing the `$log` variable with a hardcoded value such as `"Hello"`. Do you still get the error? — Code-Apprentice, Mar 19 '19 at 22:41
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/190331/discussion-between-xetra11-and-code-apprentice). — xetra11, Mar 19 '19 at 22:42
Use `jq` to build the JSON payload. Otherwise you cannot guarantee proper escaping. — bishop, Mar 19 '19 at 22:54

Charles Duffy · Accepted Answer · 2019-03-19T23:21:25.273

Use jq to generate well-formed JSON containing arbitrary contents in bash. In your use case, that might look like:

log=$(git log discord-report..development --no-merges --decorate \
  --format=format:'%C(bold green)(%ar)%C(reset) %C(white)%s%C(reset) %C(dim white)- %an')

preamble='@everyone
**Development** Branch has been updated.
Please pull the newest state!
 Changelog: '

json_payload=$(jq -nc --arg log "$log" --arg preamble "$preamble" '
{
  "username" : "CoopR Development Info",
  "content" : "\($preamble)\($log)"
}')

curl -i \
  -H "Content-Type: application/json" \
  -X POST \
  --data "$json_payload" \
  https://discordapp.com/api/webhooks/XXX/XXX

As a test anyone can run, consider the following (as of 2019-03-19 the syntax highlighting is broken, but the code works fine):

#!/usr/bin/env bash
case $BASH_VERSION in ''|[0-3]*) echo "ERROR: Bash 4.0+ needed" >&2; exit 1;; esac

log=$(cat <<'EOF'

  This is a sample message with "quotes", both 'single' and "double" and 'un'matched'.

It also has a *, and a backslash: \ <- there.
EOF
)

json_payload=$(jq -nc --arg log "$log" --arg preamble "$preamble" '
{
  "username" : "CoopR Development Info",
  "content" : "\($preamble)\($log)"
}')

printf '%s\n' "$json_payload"

...correctly emits:

{"username":"CoopR Development Info","content":"@everyone\n**Development** Branch has been updated.\nPlease pull the newest state!\n Changelog: \n  This is a sample message with \"quotes\", both 'single' and \"double\" and 'un'matched'.\n\nIt also has a *, and a backslash: \\ <- there."}

Can't use shell variable in curl data payload

What I want

1 Answers1