13

I need to send a multipart/form-data POST (xliff file) from the client to my Node.js server, and then capture the data in Node.js and forward that POST to another Java service.

I've used both multer and express-fileupload to parse the form-data stream and capture a Buffer of the xliff in Node.js and both gave me the file with its content as a buffer just fine.

However, I cannot seem to re-create a FormData object in the Node layer to forward the POST to the Java service.

I continue to get the error message "Connection terminated parsing multipart data" or just no response at all form the Java service.

I've Also attempted to use the tmp library to create a temporary file locally to write the buffer and then try to FormData('file', fs.createReadStream(<path>)), but that didn't seem to work for me either... though I'm not sure I was doing it correctly.

Using the exact same doPOST request directly form the Browser works fine, but once I try to capture the call in the Node layer and then forward the POST to the Java service, it doesn't work for me anymore.

.

const multer = require('multer');
const upload = multer();

router.post('/', upload.any(), (req, res) => {
  const { headers, files } = req;

  console.log('--------------- files:', files[0]); // object with buffer, etc.

  const XMLString = files[0].buffer.toString('utf8'); // xml string of the xliff

  const formFile = new FormData();
  formFile.append('file', XMLString);

  console.log('--------------- formFile:', formFile); // FormData object with a key of _streams: [<xml string with boundaries>, [Function: bound ]]

  headers['Content-Type'] = 'multipart/form-data';
  const url = 'some/url/to/Java/service'

  doPOST(url, formFile, {}, headers)
    .catch((error) => {
      const { status, data } = error.response;
      res.status(status).send(data);
    })
    .then(({ data }) => {
      res.send(data);
    });
});
jaimefps
  • 2,244
  • 8
  • 22
  • 45
  • You don't have to convert file buffer to xml string. Instead app directly `files[0]` as buffer array with headers `headers['Content-Type'] = 'multipart/form-data';` – front_end_dev Nov 09 '18 at 06:36
  • Passing the buffer of the xliff directly didn't work either for me. I tried that too. – jaimefps Nov 09 '18 at 16:31
  • Found an article doing the kind of same stuff. Have a look https://medium.com/technoetics/handling-file-upload-in-nodejs-7a4bb9f09a27 – front_end_dev Nov 09 '18 at 17:48
  • Thanks, already read that :) That article also refers to capturing the form-data in Node, but not how to forward the form-data to a new service, which is my problem – jaimefps Nov 09 '18 at 18:11
  • If `doPost` could accept a stream, you could stream `req` directly to the Java server, no local parsing required. – robertklep Nov 09 '18 at 19:56
  • Did you end up finding a solution to your problem? – Jean-Francois Gagnon Mar 20 '19 at 00:24

3 Answers3

11

You can directly pass the buffer to your form data, but then you also need to specify the filename parameter.

const multer = require('multer');
const upload = multer();

router.post('/', upload.any(), (req, res) => {
  const { headers, files } = req;
  const { buffer, originalname: filename } = files[0];

  const formFile = new FormData();
  formFile.append('file', buffer, { filename });

  headers['Content-Type'] = 'multipart/form-data';
  const url = 'some/url/to/Java/service'

  doPOST(url, formFile, {}, headers)
    .catch((error) => {
      const { status, data } = error.response;
      res.status(status).send(data);
    })
    .then(({ data }) => {
      res.send(data);
    });
});
Jean-Francois Gagnon
  • 3,141
  • 4
  • 20
  • 27
  • Thanks for this response! If I ever got around to working on this issue, I'll try again with your code. – jaimefps Apr 12 '19 at 16:08
  • I was trying to pass an array of files to a Spring WS using FormData and multer memoryStorage. Apparently, all I was missing was the filename in the options. Thank you for pointing that out, I couldn't see it anywhere in the documentation. Although explicitly setting the Content-Type header caused a "FileUploadException: the request was rejected because no multipart boundary was found" – Batman Aug 19 '19 at 09:15
  • Thank you. I've been trying to figure out how to use a buffer for POST and all I was missing was the `filename`. – WeerdNoise May 28 '20 at 00:31
3

This is a pseudo code example of the solution I came up with in NodeJS. I used a similar solution in ApolloGQL, but it applies all the same to ExpressJS. Hence, my example is in a pattern more akin to ExpressJS.

In the below example shows how to pass a JSON object as well as passing a file buffer into the FormData before sending it off.

const FormData = require('form-data'); // version ^3.0.0

router.post('/', async (req, res) => {
  const { body } = req;
  const { stringContentOfSomeFile } = body;
  
  // create formData for your request:
  const thisForm = new FormData();

  // passing a JSON object:
  // must declare "contentType: application/json" to avoid 415 status response from some systems:
  const someJson = JSON.stringify({  key: 'value', otherKey: 'otherValue' });
  thisForm.append('data', someJson, { contentType: 'application/json' });
  
  // passing a file buffer:
  const fileBuffer = Buffer.from(stringContentOfSomeFile, 'utf-8');
  thisForm.append('form_field_name_here', fileBuffer, 'file_name_here');

  const response = await axios.post('/path/to/endpoint', thisForm, {
    // must getHeaders() from "formData" to define the boundaries of the appended data:
    headers: { ...thisForm.getHeaders() },
  });

  // do whatever you need with the response:
  res.send(response);
});
jaimefps
  • 2,244
  • 8
  • 22
  • 45
0

For people, who is facing "FileUploadException: the request was rejected because no multipart boundary was found" this is what you need to do.

https://github.com/axios/axios/issues/1006

This resolved the issue for me. It is not axios specific, we just need to forward the formData headers that is calculated when the request is sent.

For multipart/form-data the boundary is calculated based on file content. The browser does it automagically. But in node.js we need to explicitly forward those. Thanks to form-data npm package those are done for you.