11

Is it possible to use the HTML 5 File API (for example, this library: https://github.com/23/resumable.js ) in conjunction with the S3 multi-part upload feature?

http://docs.aws.amazon.com/AmazonS3/latest/dev/uploadobjusingmpu.html

sleepy_keita
  • 1,488
  • 4
  • 17
  • 26

1 Answers1

11

Yes, but you will need some kind of server backend to handle the Amazon API keys in a more secure way if you are going to make it part of a public website.

You can find what looks like a complete example implementation of this these projects:

Please note that I have not used, tested or reviewed these projects.

A rough description of the sequence is as follows:

  1. User
    • loads webpage
    • selects file to upload
    • hits upload button
  2. Webpage
    • sends info about file to server
  3. Server
    • creates multipart upload with Amazon API
    • sends "key"(filename) and "upload id" back to Webpage
  4. Webpage
    • works out size of parts
    • requests Server to sign part passing "key", "upload id", part info
  5. Server
    • signs a part request, sends "part upload url", "date" and "auth header"
  6. Webpage
    • sends part data directly to Amazon S3 via "part upload url" using "date" and "auth header"
    • keeps track of part ids
  7. Server & Webpage
    • repeats 5 & 6 for each additional part, resuming if required
  8. Webpage
    • makes "upload complete" request to server (passing all the part info)
  9. Server
    • makes request to Amazon API to complete the creation of file
  10. Webpage
    • inform user of error or success

Notes:

  • If upload is aborted, this must also be handled server side else the parts / uploads started will be left to take up space in the S3 Bucket.
  • It might take a few minutes to complete the "upload complete" request to Amazon.
Dean Taylor
  • 40,514
  • 3
  • 31
  • 50
  • Thank you for the detailed answer! I've actually started implementing the protocol you detailed before, but haven't gotten around to actually finishing it yet: https://github.com/keichan34/s3uploader/tree/2.0-wip – sleepy_keita Oct 16 '13 at 07:28
  • 1
    Great, I've noticed you are using Ruby in your project and also found another Ruby project which I have added to the answer for your reference. – Dean Taylor Oct 16 '13 at 13:32
  • Here is another example in PHP [https://github.com/ienzam/s3-multipart-upload-browser](https://github.com/ienzam/s3-multipart-upload-browser) – roundrobin Feb 19 '14 at 20:15
  • @BausTheBig this is/was the first project linked in the answer. Obviously it wasn't clear so thanks to your comment I have updated the answer to make the link more clear. – Dean Taylor Feb 19 '14 at 22:52
  • Great thanks! The answer is very detailed, exactly what I needed right now. This is another interesting project (it uses Python) to look at: https://github.com/cinely/mule-uploader#mule-upload – roundrobin Feb 20 '14 at 01:16
  • 1
    @BausTheBig actually, the bulk of the project is written in Javascript, it only needs a small server-side endpoint, which can be written in any language easily. The "canonical" endpoint is written in Python though (Flask). – Gabi Purcaru Feb 25 '14 at 07:13