argument list too long curl

Question

Trying to solve "argument list too long" I have been searching for a solution and found the closest one to my issue curl: argument list too long however the response is not clear as I am still having the issue "argument list too long"

curl -X POST -d @data.txt \
   https://Path/to/attachments  \
   -H 'content-type: application/vnd.api+json' \
   -H 'x-api-key: KEY' \
-d '{
"data": {
  "type": "attachments",
  "attributes": {
    "attachment": {
    "content": "'$(cat data.txt | base64 --wrap=0)'",
    "file_name": "'"$FileName"'"
    }
  }
}
}'

thank you

Yeah, the answer there could use some fleshing out. Basically, you need handle the second block of data the same way as the first: put it in a file, then use `-d @filename` to tell `curl` to read it from that file. — Gordon Davisson, Aug 30 '21 at 02:27
Per the linked answer "_you are trying to pass the entirety of the base64'd content on the command line_". This is a limitation of the shell, not `curl`. The recommendation is "_curl has the ability to load in data to POST from a file_". Write the json data to some file `/tmp/data` then pass that file path to `curl` using `@` so `curl` knows it's a file path, `curl -d @/tmp/data ...`. `curl` will read data from the file `/tmp/data`. — JamesThomasMoon, Aug 30 '21 at 07:45

Léa Gris · Accepted Answer · 2021-08-31T00:15:26.413

Use jq to format your base64 encoded data string into a proper JSON string, and then pass the JSON data as standard input to the curl command.

#!/usr/bin/env sh

attached_file='img.png'

# Pipe the base64 encoded content of attached_file
base64 --wrap=0 "$attached_file" |
# into jq to make it a proper JSON string within the
# JSON data structure
jq --slurp --raw-input --arg FileName "$attached_file" \
'{
  "type": "attachments",
  "attributes": {
    "attachment": {
      "content": .,
      "file_name": $FileName
    }
  }
}
' |
# Get the resultant JSON piped into curl
# that will read the data from the standard input
# using -d @-
curl -X POST -d @- \
   'https://Path/to/attachments'  \
   -H 'content-type: application/vnd.api+json' \
   -H 'x-api-key: KEY'

JamesThomasMoon · Answer 2 · 2021-12-07T03:24:17.260

Per the linked answer

you are trying to pass the entirety of the base64'd content on the command line

This is a limitation of the shell, not curl. That is, the shell is responding with error argument list too long. The program curl is never even started.

The recommendation is

curl has the ability to load in data to POST from a file

Write the json data to some file /tmp/data.json using piping.
(the commands will use piping | and file redirection > >> which can handle arbitrarily large amounts of data. Whereas, you cannot place arbitrarily large amounts of data into a single command, there is a limit).

echo -n '
{
"data": {
  "type": "attachments",
  "attributes": {
    "attachment": {
    "content": "' > /tmp/data.json

cat data.txt | base64 --wrap=0 >> /tmp/data.json

echo -n '",
    "file_name": "'"$FileName"'"
    }
  }
}
}' >> /tmp/data.json

Pass that file path /tmp/data.json to the curl command using @ so curl knows it's a file path.

curl -X POST -d @/tmp/data.json \
   "https://Path/to/attachments"  \
   -H 'content-type: application/vnd.api+json' \
   -H 'x-api-key: KEY'

argument list too long curl

2 Answers2