3

The Problem

I've been trying to set the size parameter in the content-disposition header for quite some time.

function uploadFile(file) {
    const data = new FormData();
    data.append('file', file);

    axios.post('url/of/endpoint', data);

FormData does not send the size parameter by default and, to my knowledge, I can't see a way of setting it myself. Is there a way of doing this with Axios? Do I need to go more lower level and build my own headers?

The Context

My primary objective is to pipe a file input stream via a Java endpoint to S3. But without the file size, all of the data gets loaded into memory and has a high chance of causing an OOM exception. This specific problem has been asked here as well. Is writing my own byte buffer the only way? Should I just stop being lazy and do that? Seems like an overly-complex solution...

Is there a reason as to why the file size isn't normally included in FormData? I've tried using Content-Length, but that includes the content-disposition header and leads to a mismatched byte count.

Kresimir
  • 777
  • 5
  • 20
Wompinalong
  • 143
  • 1
  • 8
  • What would you do with `size` or `content-length`? – Kosh Dec 06 '18 at 21:00
  • @KoshVery Give it to the Amazon SDK so it can break down the file for multipart? Again, [the context has been asked here](https://stackoverflow.com/questions/8653146/can-i-stream-a-file-upload-to-s3-without-a-content-length-header). – Wompinalong Dec 06 '18 at 21:05
  • Is it possible if storing in cloud dB instead of cloud file storage https://www.google.com/amp/s/amp.reddit.com/r/aws/comments/5s6g3n/binaryblob_storage_alternative_in_aws/#ampf=undefined – estinamir Dec 06 '18 at 21:27
  • Java or JavaScript? – Barmar Dec 06 '18 at 21:45
  • @Barmar original question is for Javascript, the context involves Java – Wompinalong Dec 06 '18 at 21:46

1 Answers1

1

Right, so after going around the office... then to my friends... I come to the conclusion that this is a Java shortcoming. Well, not the original question, but the fact that a multipart upload requires the total file size. So, the implementation I'm currently going with, which makes me feel super dirty, is adding a FormData item size on the client side.

function uploadFile(file) {
    const data = new FormData();
    data.append('size', file.size.toString());
    data.append('file', file);

    axios.post('url/of/endpoint', data);

And then updating the backend to pass that size to the S3 SDK... ... oof.

Wompinalong
  • 143
  • 1
  • 8