Create JSON Gist (conflict)

Question

I want to create a Gist containing a JSON (valid one, I checked) with curl command as described here.

I tried first this script :

configText=$(cat jsonFile.json)

generate_post_data()
{
cat <<EOF
{
  "description": "the description for this gist",
  "public": true,
  "files": {
    "file1.txt": {
      "content": $configText
    }
  }
}
EOF
}

curlBody="$(generate_post_data)"

curlResponse=$(curl -H "Content-Type: application/json" -X POST -d '$curlBody' https://api.github.com/gists)

Which gave me the error Problems parsing JSON, so I tried passing the file directly in the command:

curl -H "Content-Type:application/json" -data-binary @jsonFile.json https://api.github.com/gists

But I'm getting the same error. I know that this must be a conflict between the JSON body of the POST request and the JSON of my file (quotes, brackets...).

How can I send a clean JSON file to Gist ?

score 2 · Accepted Answer · edited May 23 '17 at 12:17

For the issues in your script :

in your curl request, you use single quotes around your bash variable in POST -d '$curlBody', use double quote to expand it : POST -d "$curlBody"
content is a text field : "content": $configText to "content": "$configText"
configText can have new lines and unescaped double quotes " which break your content JSON data. You could use the following to escape quotes and remove new lines :
```
configText=$(cat test.json | sed 's/\"/\\\"/g' | tr -d '\n')
```

The following example build your gist request with jq JSON parser/builder, not that this example will not preserve new lines in your input :

#!/bin/bash

ACCESS_TOKEN="YOUR_ACCESSS_TOKEN"

description="the description for this gist"
filename="file1.txt"

curlBody=$(jq --arg desc "$description" --arg filename "$filename"  '.| 
{ "description": $desc, 
  "public": true, 
  "files": { 
      ($filename) : { 
          "content": tostring 
       } 
   } 
}' jsonFile.json)

curl -v -H "Content-Type: application/json" \
        -H "Authorization: Token $ACCESS_TOKEN" \
        -X POST -d "$curlBody" https://api.github.com/gists

The following will preserve new lines in your json input by replacing new lines with \\n :

#!/bin/bash

ACCESS_TOKEN="YOUR_ACCESSS_TOKEN"

description="the description for this gist. There are also some quotes 'here' and \"here\" in that description"
public="true"
filename="file1.txt"

desc=$(echo "$description" | sed 's/"/\\"/g' | sed ':a;N;$!ba;s/\n/\\n/g')
json=$(cat test.json | sed 's/"/\\"/g' | sed ':a;N;$!ba;s/\n/\\n/g')

curl -v -H "Content-Type: text/json; charset=utf-8" \
        -H "Authorization: Token $ACCESS_TOKEN" \
        -X POST https://api.github.com/gists -d @- << EOF
{ 
  "description": "$desc", 
  "public": "$public", 
  "files": { 
      "$filename" : { 
          "content": "$json"
       } 
   } 
}
EOF

Note that your access token must have the gist scope

I've updated my post with a version that is preserving new lines in your gist content — Bertrand Martel, Feb 24 '17 at 21:15

Create JSON Gist (conflict)

1 Answers1