1

I read a number of posts, including Amazon example copied into https://stackoverflow.com/a/17664640/308851 Direct uploads to AWS S3 from the browser and more.

None of this covers more than just a single file -- but who wants a piece of file without metadata? I'd prefer to capture all of the user upload , files (multiple) and form fields and all and just send them JSON encoded but I can't find anything on how to do this.

chx
  • 11,270
  • 7
  • 55
  • 129

1 Answers1

0

Each request to S3 only involves one object at a time. Normally, each file in the form would be a separate PUT request, and the rest of the form data would be separate as well.

While you definitely can combine all your data into one thing, you really don't want JSON for this. The reason is that JSON is a text-based format, so the only way you'll be able to put binary data in it is by encoding it as base64, increasing the size by 33%. Making matters worse, the entire contents of the file would be in memory the entire time, possibly several times at once while you try to JSON encode the object. Then, it has to be uploaded. Don't use JSON.

There are alternatives to JSON such as CBOR, which are easily handled through JavaScript. These would allow you to bundle all your files and form data into one blob and upload without needing to base64 encode. However, you still have the problem that your upload to S3 is not inherently streamable from a browser while the encoding is occurring, so everything would sit in memory.

You have two choices:

  • Option A: Proxy the data
    In this case, you handle the form POST on your own application server, split the form data up, and then upload it to S3 from your server. As you know, this is a waste of bandwidth.
  • Option B: Multiple requests to S3
    You'll have to write some code to handle potentially invalid states (such as partial form uploads), but you'll be able to make several PUT requests and get your data where it needs to go, without proxying the data through your server. This is by far the most efficient and practical way to go.
Brad
  • 159,648
  • 54
  • 349
  • 530