Curl: how to pass value of specific key from file

Question

wasn't able to find and I'm not sure if it possible at all. I know that I can pass the whole payload via @file.json, something like

$ curl -X POST -H "Content-Type: application/json" -d @test.json \
http://localhost/api/v3/endpoint 

$ cat test.json
{
   "key1": "value1",
   "key2": "value2"
}

But now I need to upload gpg public key and I can't store the key inside test.json. What I need is something like

$ curl -X POST -H "Content-Type: application/json" \
http://localhost/api/v3/endpoint \
-d '{"key1": "value1", "key2": "@gpg-public.key"}'

Thanks in advance

As an option in bash I can use the following

$ curl -X POST -H "Content-Type: application/json" \
http://localhost/api/v3/endpoint \
-d "{\"key1\": \"value1\", \"key2\": \"$(cat gpg-public.key)\"}"

but I'm looking for more elegant way

unless you can guarantee that the gpg file doesn't contain `\n` (it almost certainly does!!) or `"`, or anything else that needs to be json-escaped , your proposed bash cat method is not safe, it will create a corrupted/invalid json. — hanshenrik, Jun 15 '18 at 12:03

score 3 · Answer 1 · answered Jun 15 '18 at 10:25

3

I can't say this is more elegant, but at least it's different.

jq -Rn '{key2: input}' <pgp-public.key |
jq -s '.[0] * .[1]' test.json - |
curl -d @- ...

Credit for the jq hacks: Simo Kinnunen's answer here and Charles Duffy's answer here.

answered Jun 15 '18 at 10:25

tripleee

175,061
34
275
318

3

Put a `jq` filter based on `test.json`, like `{key1: "value1", key2: $pgp}`, in a file, and then you can make a single `jq` call: `jq -n --arg pgp "$( – chepner Jun 15 '18 at 13:55
@chepner what is ```-f test.jq``` or it's just a typo and should be test.json? – ALex_hha Jun 15 '18 at 16:17
`-f` tells `jq` to read the filter from the given file, rather than from a command line argument. If `test.jq` contains `.`, then `jq -f test.jq` is equivalent to `jq '.'`. – chepner Jun 15 '18 at 16:21
But if `test.json` is a given and cannot be modified, putting `key2: $pgp` inside it is a non-starter. – tripleee Jun 15 '18 at 18:20

score 1 · Accepted Answer · answered Jun 15 '18 at 16:29

The first rule of JSON is to use a tool that knows JSON to generate dynamic values. A good tool is jq. Now, instead of storing a JSON file on disk, store a jq filter that can act as a template for your JSON file.

# test.jq
{
  key1: "value1",
  key2: $gpg
}

Now, you can run jq using this template as a filter, and supply a value for it to replace $gpg:

$ jq -n -f test.jq --arg gpg hi
{
  "key1": "value1",
  "key2": "hi"
}

Once you have this command, you can use it to pass the generated JSON directly to curl via standard input; the -d option can take @- as an argument to read from standard input.

jq -n -f test.q --arg gpg hi | curl ... -d @-

For simple jobs, you can specify the filter on the command line instead of reading it from a file.

jq -n '{key1: "value1", key2: $gpg}' --arg gpg hi | curl -d @-

hanshenrik · Answer 3 · 2018-06-15T11:45:31.033

use multipart/form-data with the -F parameter, it supports uploading multiple files at once.

curl -F json=@test.json -F pgp=@pgp-public.key http://localhost/api/v3/endpoint

ofc, the API endpoint will also need to support multipart/form-data for this to work.

edit: going by your comments, if you have to send a single json file, then edit the json file as needed and send the edited version, unfortunately bash is not a suitable language for editing json (although i'm sure it's possible), i'd recommend using some other scripting language for that. here's an example with php-cli to edit the json, then sending the edited json to curl via stdin:

php -r '$p=json_decode(file_get_contents("test.json"),true);$p["key2"]=file_get_contents("pgp-public.key");echo json_encode($p);' | 
curl -d @- -X POST -H "Content-Type: application/json" http://localhost/api/v3/endpoint

(but any language with json support should suffice, like python/perl/php)

@ALex_hha then use a better scripting language. `php -r '$p=json_decode(file_get_contents("test.json"),true);$p["key2"]=file_get_contents("pgp-public.key");echo json_encode($p);' | curl -d @- -X POST -H "Content-Type: application/json" http://localhost/api/v3/endpoint` — hanshenrik, Jun 15 '18 at 09:43

Curl: how to pass value of specific key from file

3 Answers3