bash: Getting error while executing curl request

Question

I am trying to execute a bash script but I am getting this error:

curl: (3) URL using bad/illegal format or missing URL
curl: (3) unmatched close brace/bracket in URL position 1

I have tried several things, like escaping the " in json, using -g in curl request, using `` around Req variable instead of using eval, but I am unable to fix this.

   #!/bin/bash
filename="$1"
URL='http://localhost:8983/contentCleaner?action=clean'
while read -r requestParam; do
   Req="curl ${URL} -X POST -H 'Accept:application/json' -H 'Content-Type:application/json' --data \"$requestParam\""
    Res=eval $Req
    echo "Res:$Res"
done < "$filename"

The file I am passing as param to the bash command contains a list of json data which is separated by new line. The bash script loops through the json and populates the curl request. Here is a sample of one request:

{ \"batchRequest\": { \"requests\": [ { \"contentData\": { \"contentText\": { \"ingestedContentSummary\": { \"provider\": { \"url\": \"twitter.com\" }, \"title\": \"Trump\" } } }, \"contentSource\": \"twitter\", } ] } }

Have you tried double quotes around `${URL}`? `Req="curl \"${URL}\" ...` — mouviciel, Nov 02 '22 at 10:42
@mouviciel yes i have tried that but still got the same error — isha chauhan, Nov 02 '22 at 11:09
@mouviciel: The variable `URL` does not contain spaces; double-quoting can't have any effect. — user1934428, Nov 02 '22 at 13:09
@user1934428 - Agreed. I'm used to quote URLs with curl because of `&` characters, which are absent here. — mouviciel, Nov 02 '22 at 14:08

score 0 · Accepted Answer · answered Nov 02 '22 at 11:48

Don't (try to) put shell commands in (scalar) variables. It only works for the very simplest cases, and your case isn't very simple. We have lots of previous Qs with this problem, but unfortunately they're hard to search for. Instead as a reference see https://mywiki.wooledge.org/BashFAQ/050 .

For your case, just execute the correct command:

while read -r requestParam; do
   Res=$( curl ${URL} -H 'Accept:application/json' -H 'Content-Type:application/json' --data "$requestParam" ) # -X POST 
    echo "Res:$Res"
done < $filename

You don't need -X POST because all --data* options automatically set it. Also, <$var (and >$var >>$var <<$var <<<$var) are not subject to wordsplitting and globbing so don't need double-quoting. (In contrast "$requestParam" as a command argument does.)

Or since the only thing you are doing with the result is outputting it with a prefix, you can simplify to

while read -r requestParam; do
   curl ${URL} -H 'Accept:application/json' -H 'Content-Type:application/json' --data "$requestParam" | sed 's/^/Res:/'
done <$filename

and if each body has a newline (like your input file) further to

xargs <$filename -I{} curl $URL -H... -H... --data {} | sed 's/^/Res:/'

You still need double quotes around the URL, otherwise this will fail if it contains e.g. `&`. See also [When to wrap quotes around a shell variable](https://stackoverflow.com/questions/10067266/when-to-wrap-quotes-around-a-shell-variable) — tripleee, Nov 02 '22 at 12:07
Maybe it's worth mentioning that if the OP **does** want to use a variable for holding the command, and array might help. — user1934428, Nov 02 '22 at 13:10

bash: Getting error while executing curl request

1 Answers1