91

I have problem uploading file using POST request in Node.js. I have to use request module to accomplish that (no external npms). Server needs it to be multipart request with the file field containing file's data. What seems to be easy it's pretty hard to do in Node.js without using any external module.

I've tried using this example but without success:

request.post({
  uri: url,
  method: 'POST',
  multipart: [{
    body: '<FILE_DATA>'
  }]
}, function (err, resp, body) {
  if (err) {
    console.log('Error!');
  } else {
    console.log('URL: ' + body);
  }
});
Philip Kirkbride
  • 21,381
  • 38
  • 125
  • 225
Łukasz Jagodziński
  • 3,009
  • 2
  • 25
  • 33

6 Answers6

134

Looks like you're already using request module.

in this case all you need to post multipart/form-data is to use its form feature:

var req = request.post(url, function (err, resp, body) {
  if (err) {
    console.log('Error!');
  } else {
    console.log('URL: ' + body);
  }
});
var form = req.form();
form.append('file', '<FILE_DATA>', {
  filename: 'myfile.txt',
  contentType: 'text/plain'
});

but if you want to post some existing file from your file system, then you may simply pass it as a readable stream:

form.append('file', fs.createReadStream(filepath));

request will extract all related metadata by itself.

For more information on posting multipart/form-data see node-form-data module, which is internally used by request.

Leonid Beschastny
  • 50,364
  • 10
  • 118
  • 122
  • 91
    When I was learning node and the request module, I was confused as to why the form could be modified after the `post` method was called. Buried in the [request](https://github.com/request/request) docs is the explanation - the form "_can be modified until the request is fired on the next cycle of the event-loop_". – Doug Donohoe Mar 23 '15 at 15:08
  • 3
    I keep getting '[Error: write after end]' when using form and form.append, anyone knows why? – Vitor Freitas Feb 19 '16 at 13:02
  • 1
    @VitorFreitas you should call `req.form()` and fill it with all appropriate data synchronously right after calling `request.post`. It's important to do it during the same event loop tick, otherwise your request may be already send and underlying stream closed. – Leonid Beschastny Feb 19 '16 at 13:30
  • @LeonidBeschastny Can you take a look into my code? http://pastebin.com/E6b0cvag . It seems right to me, thanks! requestService is the request module – Vitor Freitas Feb 19 '16 at 13:56
  • @VitorFreitas looks fine to me, too. – Leonid Beschastny Feb 19 '16 at 14:17
  • Does anyone have an answer to the Error: write after end ? I have the same issue and I am sure that I am adding to the form on the same event tick. – nbroeking Jun 14 '16 at 15:45
  • This causes the same 'Error: write after end' error for me. Cannot find a working example of posting multipart using request. – Elliot Jun 27 '16 at 18:21
  • `request will extract all related metadata by itself.` wait, are you saying it will add fields to the form, assuming it knows what the server wants? – Michael Sep 29 '16 at 20:10
  • @Michael I mean that it will automatically include all related file metadata like filename, content type and content length. – Leonid Beschastny Sep 30 '16 at 09:55
  • This works for me, and introduced me to a new useful lib, AWESOME! – nagates Apr 28 '17 at 22:13
  • Even I have the same issue of `Error: write after end`. And even I'm doing stuff in the same process tick. – Asif Ali Apr 03 '18 at 10:22
  • I'm trying to send a PDF with `form.append('file', fs.createReadStream(pathToPDF));`, but the PDF ends damaged. Thats because somehow some meta data gets added to the beginning and those bytes are missing at the end. I have found, that there is a `_streams` Array in the FormData Object, that might causes the Problem. The Array starts with those metadata as Strings, followed by the ReadStream Object itself. Is it possible to delete those metadata? Or instead of appending the file to the key `file` setting it directly? – Seb Jan 03 '20 at 12:41
  • @SebastianJ.Vogt are you referring to `multipart/form-data` request metadata? Are you sure you need a `multipart/form-data` request instead of sending a single file in HTTP body? – Leonid Beschastny Jan 04 '20 at 17:22
  • Yes I am. It's how the Microsoft API expects the data. – Seb Jan 05 '20 at 08:04
  • I keep getting "Error: MultipartParser.end(): stream ended unexpectedly: state = PART_DATA". anyone knows how to fix this. – Amy Doxy Apr 22 '20 at 09:09
  • 1
    The [request](https://github.com/request/request) was deprecated, do you have a alternative? – David Sep 14 '20 at 20:57
  • 1
    @David [got](https://www.npmjs.com/package/got) is a good alternative – Leonid Beschastny Sep 15 '20 at 11:00
22

An undocumented feature of the formData field that request implements is the ability to pass options to the form-data module it uses:

request({
  url: 'http://example.com',
  method: 'POST',
  formData: {
    'regularField': 'someValue',
    'regularFile': someFileStream,
    'customBufferFile': {
      value: fileBufferData,
      options: {
        filename: 'myfile.bin'
      }
    }
  }
}, handleResponse);

This is useful if you need to avoid calling requestObj.form() but need to upload a buffer as a file. The form-data module also accepts contentType (the MIME type) and knownLength options.

This change was added in October 2014 (so 2 months after this question was asked), so it should be safe to use now (in 2017+). This equates to version v2.46.0 or above of request.

Clavin
  • 1,182
  • 8
  • 17
4

Leonid Beschastny's answer works but I also had to convert ArrayBuffer to Buffer that is used in the Node's request module. After uploading file to the server I had it in the same format that comes from the HTML5 FileAPI (I'm using Meteor). Full code below - maybe it will be helpful for others.

function toBuffer(ab) {
  var buffer = new Buffer(ab.byteLength);
  var view = new Uint8Array(ab);
  for (var i = 0; i < buffer.length; ++i) {
    buffer[i] = view[i];
  }
  return buffer;
}

var req = request.post(url, function (err, resp, body) {
  if (err) {
    console.log('Error!');
  } else {
    console.log('URL: ' + body);
  }
});
var form = req.form();
form.append('file', toBuffer(file.data), {
  filename: file.name,
  contentType: file.type
});
Philip Kirkbride
  • 21,381
  • 38
  • 125
  • 225
Łukasz Jagodziński
  • 3,009
  • 2
  • 25
  • 33
  • 4
    There is a simpler way to convert `ArrayBuffer` to `Buffer`, using build-in `Buffer` [constructor from an array of octets](http://nodejs.org/api/buffer.html#buffer_new_buffer_array): `var buffer = new Buffer(new Uint8Array(ab));` – Leonid Beschastny Aug 17 '14 at 11:02
  • 2
    Where did the "file" in file.data, file.name, and file.type come from in your last function? I don't see that variable mentioned anywhere else. – michaelAdam Jul 02 '15 at 23:58
  • I'm using Meteor and community package for file management. However if you are using pure node then you can use file system functions to get all the info about the file and its data https://nodejs.org/api/fs.html – Łukasz Jagodziński Jul 03 '15 at 07:23
4

You can also use the "custom options" support from the request library. This format allows you to create a multi-part form upload, but with a combined entry for both the file and extra form information, like filename or content-type. I have found that some libraries expect to receive file uploads using this format, specifically libraries like multer.

This approach is officially documented in the forms section of the request docs - https://github.com/request/request#forms

//toUpload is the name of the input file: <input type="file" name="toUpload">

let fileToUpload = req.file;

let formData = {
    toUpload: {
      value: fs.createReadStream(path.join(__dirname, '..', '..','upload', fileToUpload.filename)),
      options: {
        filename: fileToUpload.originalname,
        contentType: fileToUpload.mimeType
      }
    }
  };
let options = {
    url: url,
    method: 'POST',
    formData: formData
  }
request(options, function (err, resp, body) {
    if (err)
      cb(err);

    if (!err && resp.statusCode == 200) {
      cb(null, body);
    }
  });
chrismarx
  • 11,488
  • 9
  • 84
  • 97
  • 6
    Please [edit] your answer and add some explanation or comment as to how your code works. This would help other users decide whether your answer is interesting enough to be considered. Otherwise people have to analyse your code (which takes time) even to have a vague idea whether this might be what they need. Thank you! – Fabio says Reinstate Monica Mar 25 '17 at 17:40
  • 5 years later someone is going to want an explanation and you're not going to be around or not going to bother. That's why Fabio asked you to put the explanation in the response, and not on request. – user985366 Aug 08 '17 at 11:24
0

I did it like this:

// Open file as a readable stream
const fileStream = fs.createReadStream('./my-file.ext');

const form = new FormData();
// Pass file stream directly to form
form.append('my file', fileStream, 'my-file.ext');
Atul
  • 3,778
  • 5
  • 47
  • 87
-1
 const remoteReq = request({
    method: 'POST',
    uri: 'http://host.com/api/upload',
    headers: {
      'Authorization': 'Bearer ' + req.query.token,
      'Content-Type': req.headers['content-type'] || 'multipart/form-data;'
    }
  })
  req.pipe(remoteReq);
  remoteReq.pipe(res);