4

There are similar questions like this, this, this, and this, but none help.

In Node, the goal is to use the axios module to download an image from Twitter then upload this image as an image file as part of a form submission.

This code downloads the Twitter image, but the uploaded binary image is corrupted when it reaches the server (also our server).

The image must be uploaded in binary.

A form is required because other fields are also submitted with the binary image. These other form fields were removed to simplify the code sample.

        const axios = require('axios');
        const FormData = require('form-data');

        let response = await axios.get('https://pbs.twimg.com/media/EyGoZkzVoAEpgp9.png', {
                                responseType: 'arraybuffer'
                             });

        let imageBuffer = Buffer.from(response.data, 'binary');    
        let formData = new FormData();          
        formData.append('image', imageBuffer);

        try {
            let response = await axios({
                method: 'post',
                url: serverUrl,
                data: formData,
            });

            // Do stuff with response.data

        } catch (error) {
            console.log(error)
        }
Crashalot
  • 33,605
  • 61
  • 269
  • 439
  • Can you clarify what you mean by "something fails between this step and performing the upload"? Can you share the error message? – Arun Kumar Mohan Apr 23 '21 at 11:19
  • @ArunKumarMohan thanks for the reply. Updated the question: the image gets uploaded in a corrupted state. (Also the image must be uploaded in binary.) – Crashalot Apr 23 '21 at 15:18
  • Well how are you reading this data on the server? It seems odd to try and append the binary data into a form. Form post data won't typically accept this but, well it depends on how your server is processing the data – Liam Apr 23 '21 at 15:23
  • I'd suggest something more like this https://stackoverflow.com/a/50543235/542251 – Liam Apr 23 '21 at 15:24
  • @Liam thanks for the reply. Updated the question: a form is used because we submit other data along with the binary image. The other fields were removed in the code sample to simplify the code. Given that a form is required, do you have any suggestions? Thanks for your help! – Crashalot Apr 23 '21 at 16:26
  • @Crashalot Can you see if adding `headers: formData.getHeaders()` to the Axios call changes anything? – Arun Kumar Mohan Apr 23 '21 at 16:54
  • @ArunKumarMohan thanks for the idea, but it did not. any other suggestions? does it seem like the code should work? – Crashalot Apr 24 '21 at 06:37
  • Hi @ArunKumarMohan do you happen to have other suggestions? Are you available to offer Node consulting? – Crashalot Apr 25 '21 at 20:23
  • @Crashalot "do you happen to have other suggestions?" Can you update your question with a minimal reproducible script using a fake API or something? I can take a look when I get a chance. – Arun Kumar Mohan Apr 25 '21 at 20:37
  • @ArunKumarMohan sure, could I email you as the fake API would contain private details? – Crashalot Apr 25 '21 at 21:10
  • 1
    @Crashalot I was able to reproduce the issue using [httpbin](https://httpbin.org/) as the API. – Arun Kumar Mohan Apr 26 '21 at 00:26
  • Have you tried using a tool like Fiddler to look at the HTTP trace? – Alex Riveron Apr 26 '21 at 00:38

2 Answers2

9

You should pass the headers to the axios call using formData.getHeaders() to send a Content-Type header of multipart/form-data. Without it, a Content-Type header of application/x-www-form-urlencoded is sent. You could pass a responseType of stream to the axios call that downloads the image and add the stream to the form data.

You can also use axios.post to simplify the method call.

const url = 'https://pbs.twimg.com/media/EyGoZkzVoAEpgp9.png'
const { data: stream } = await axios.get(url, {
  responseType: 'stream',
})

const formData = new FormData()
formData.append('image', stream)

try {
  const { data } = await axios.post('http://httpbin.org/post', formData, {
    headers: formData.getHeaders(),
  })
  console.log(data)
} catch (error) {
  // handle error
}

Edit Axios File Upload With Streams

Arun Kumar Mohan
  • 11,517
  • 3
  • 23
  • 44
  • Should the input type be 'file' instead of 'image'? – Alex Riveron Apr 26 '21 at 00:42
  • @AlexRiveron Sorry, not sure if I understand. Are you referring to `'image'` passed to `formData.append`? If yes, it is the key and it could be any string. You can check the docs [here](https://github.com/form-data/form-data#void-append-string-field-mixed-value--mixed-options-). – Arun Kumar Mohan Apr 26 '21 at 00:44
  • Sorry Arun, that was my misunderstanding. – Alex Riveron Apr 26 '21 at 00:57
  • @AlexRiveron No worries. – Arun Kumar Mohan Apr 26 '21 at 00:58
  • Is the `stream` variable that you're passing to `formData.append` considered a `USVString` or `Blob`? According to the Mozilla Developer [docs](https://developer.mozilla.org/en-US/docs/Web/API/FormData/append#append_parameters), if it's not either one of those or a subclass, it's converted to a string. – Alex Riveron Apr 26 '21 at 01:00
  • 1
    @AlexRiveron It's a readable stream. We're in Node land, btw. So, `USVString` and `Blob` don't exist. You could fork the linked CodeSandbox and play with it. – Arun Kumar Mohan Apr 26 '21 at 01:02
  • Thanks, will try this right now! You're awesome. – Crashalot Apr 26 '21 at 01:41
0

You can use the fetch API to fetch the image as a blob object and append it to form data. Then simply upload it to its destination using Axios, ajax, or the fetch API:

    const mediaUrl = "https://pbs.twimg.com/media/EyGoZkzVoAEpgp9.png"
    fetch(mediaUrl)
      .then((response) => response.blob())
      .then((blob) => {
        // you can also check the mime type before uploading to your server
        if (!['image/jpeg', 'image/gif', 'image/png'].includes(blob?.type)) {
          throw new Error('Invalid image');
        }

        // append to form data
        const formData = new FormData();
        formData.append('image', blob);

        // upload file to server
        uploadFile(formData);
      })
      .catch((error) => {
        console.log('Invalid image')
      });
Abulo John Joshua
  • 373
  • 4
  • 11