Why does this curl command return Unexpected EOF

Question

I am somewhat new to Bash and cURL and cannot understand out why this Bash file does not run without throwing an Unexpected EOF error.

This cURL command should upload a large file (in the example script below, somewhere between 300 and 400 MB) in 20 MB chunks to a storage service. Once all MB are uploaded, a second command "completes" the upload. Both commands use the same GUID.

Inside upload-bits.sh:

#!/bin/sh
for i in {0..20}; do
curl -X POST \
  https://community.<company>.com/api.ashx/v2/cfs/temporary.json \
  -H 'Rest-User-Token: 12345' \
  -F UploadContextId=21f23109-aac2-44ef-8b89-c0f62e67da4d \
  -F FileName='file.zip' \
  -F TotalChunks=20 \
  -F CurrentChunk=$i \
  -F 'file=@file.zip'
done

The Bash script throws the Unexpected EOF error. I have tried the cURL command alone without the Bash portion of the script and replaced CurrentChunk with 0 and 1 without success. I also used a script validator, which confirmed there were no problems in the script. I also ran dos2unix on it in a desire to eliminate end-of-line issues.

I have not been able to use this second script yet, as the first script has not worked, but I am posting it for context if I am not explaining the desired overall process well.

complete-upload.sh:

curl -X POST \
  https://community.<company>.com/api.ashx/v2/media/371/files.json \
  -H 'Rest-User-Token: 12345' \
  -F 'Name=file.zip' \
  -F ContentType=application/zip \
  -F FileName='file.zip' \
  -F FileUploadContext=21f23109-aac2-44ef-8b89-c0f62e67da4d

I would be grateful for any tips or insights. Thank you.

What's the *exact* error? `Unexpected EOF` doesn't look like a `bash` error message. — chepner, Feb 06 '19 at 21:42
Sorry @chepner I should have provided that. It is `curl: (56) Unexpected EOF`. I assumed `56` was a line ending somewhere (my script ends at line 11) so I wasn't sure. — hcdocs, Feb 06 '19 at 21:46
No, that's an error message directly from `curl`. I can only assume there is a problem with `file.zip`. (Or possibly a network communication error? 56 appears to be the exit status; from the man page, "56 Failure in receiving network data.") — chepner, Feb 06 '19 at 21:48
EOF=End Of File : Curl expects more data, because some data/structures were not completed/closed. — Wiimm, Feb 06 '19 at 21:48
Every chunk is chunk 0 - that seems wrong. Surely that should be `$i`? — l0b0, Feb 06 '19 at 21:54
@l0b0 You are right - I forgot to change it back after testing. I edited the question to reflect the $i. Thank you for calling that out — hcdocs, Feb 06 '19 at 23:37
also it is a good practice to enclose variables like $i in quotes so, makes sure to keep it as "$i" not $i.. — ArigatoManga, Feb 07 '19 at 00:39
You appear to be sending file.zip exactly twenty times (`for i in {1..20}`) but it's not clear where you are splitting it into 20MB chunks. — jhnc, Feb 10 '19 at 04:50

jhnc · Accepted Answer · 2019-02-10T06:48:05.243

Judging by the parameters passed to curl, the server expects chunked data.

However the curl command sends the whole file 20 times.

Looking at definition of CurrentChunk at https://community.telligent.com/community/10/w/api-documentation/61481/upload-cfs-rest-endpoint , perhaps a modification like this would work:

#!/bin/bash

# using GNU split options will make arithmetic simpler
# with -d, we may get numbers like 09 which are invalid octal
# start from 101 if CurrentChunk is one-based
# start from 100 if CurrentChunk is zero-based
split -b20M -a3 --numeric-suffixes=101 file.zip part.

partlist=( part.* )
numparts=${#partlist[@]}

for part in ${partlist[@]}; do
  i=$(( ${part##*.}-100 ))
  curl -X POST \
    https://community.<company>.com/api.ashx/v2/cfs/temporary.json \
    -H 'Rest-User-Token: 12345' \
    -F UploadContextId=21f23109-aac2-44ef-8b89-c0f62e67da4d \
    -F FileName='file.zip' \
    -F TotalChunks=$numparts \
    -F CurrentChunk=$i \
    -F 'file=@'$part
done

rm ${partlist[@]}

This is the answer - thank you. I did not realize the file had to be split. The only issue I have left is that it works when the file can be uploaded in 1 part but not when the file requires multiple parts. But I did not provide much detail to go on so thank you for pointing me in the right direction. — hcdocs, Feb 11 '19 at 20:31

Why does this curl command return Unexpected EOF

1 Answers1