How to Upload files to Amazon AWS3 with NodeJS?

Question

I'm trying to upload files from a MERN application I'm working on. I'm almost done with the NodeJS back end part.

Said application will allow users to upload images(jpg, jpeg, png, gifs, etc) to an Amazon AWS S3 bucket that I created.

Well, lets put it this way. I created a helper:

const aws = require('aws-sdk');
const fs = require('fs');

// Enter copied or downloaded access ID and secret key here
const ID = process.env.AWS_ACCESS_KEY_ID;
const SECRET = process.env.AWS_SECRET_ACCESS_KEY;

// The name of the bucket that you have created
const BUCKET_NAME = process.env.AWS_BUCKET_NAME;

const s3 = new aws.S3({
  accessKeyId: ID,
  secretAccessKey: SECRET
});

const uploadFile = async images => {
  // Read content from the file
  const fileContent = fs.readFileSync(images);

  // Setting up S3 upload parameters
  const params = {
    Bucket: BUCKET_NAME,
    // Key: 'cat.jpg', // File name you want to save as in S3
    Body: fileContent
  };

  // Uploading files to the bucket
  s3.upload(params, function(err, data) {
    if (err) {
      throw err;
    }
    console.log(`File uploaded successfully. ${data.Location}`);
  });
};

module.exports = uploadFile;

That helper takes three of my environment variables which are the name of the bucket, the keyId and the secret key.

When adding files from the form(that will eventually be added in the front end) the user will be able to send more than one file.

Right now my current post route looks exactly like this:

req.body.user = req.user.id;
req.body.images = req.body.images.split(',').map(image => image.trim());
const post = await Post.create(req.body);

res.status(201).json({ success: true, data: post });

That right there works great but takes the req.body.images as a string with each image separated by a comma. What would the right approach be to upload(to AWS S3) the many files selected from the Windows directory pop up?. I tried doing this but did not work :/

// Add user to req,body
req.body.user = req.user.id;
uploadFile(req.body.images);
const post = await Post.create(req.body);

res.status(201).json({ success: true, data: post });

Thanks and hopefully your guys can help me out with this one. Right now I'm testing it with Postman but later on the files will be sent via a form.

score 0 · Answer 1 · answered Nov 22 '19 at 15:19

Well you could just call the uploadFile multiple times for each file :

try{
    const promises= []
    for(const img of images) {
          promises.push(uploadFile(img))
        }
    await Promise.all(promises)
    //rest of logic

}catch(err){ //handle err }

On a side note you should warp S3.upload in a promise:

const AWS = require('aws-sdk')

const s3 = new AWS.S3({
  accessKeyId: process.env.AWS_ACCESS_KEY,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
})

module.exports = ({ params }) => {
  return new Promise((resolve, reject) => {
    s3.upload(params, function (s3Err, data) {
      if (s3Err) return reject(s3Err)
      console.log(`File uploaded successfully at ${data.Location}`)
      return resolve(data)
    })
  })
}

Bonus, if you wish to avoid having your backend handle uploads you can use aws s3 signed urls and let the client browser handle that thus saving your server resources.

One more thing your Post object should only contain Urls of the media not the media itself.

Your code throws me an error of 'images is not defined'. I tried using const images = req.body.images but did not work. — Kirasiris, Nov 22 '19 at 23:42
the code is just for reference, I don't know how you're sending your data to the server. req.body.images should be an array of images. (My guess would be that you base 64 encode binary data and send it). This might help, https://stackoverflow.com/questions/26805005/sending-binary-data-to-amazon-s3-javascript — B.Mouad, Nov 23 '19 at 09:56

score 0 · Answer 2 · answered Dec 06 '19 at 09:22

// Setting up S3 upload parameters
    const params = {
        Bucket: bucket, // bucket name
        Key: fileName, // File name you want to save as in S3
        Body: Buffer.from(imageStr, 'binary'), //image must be in buffer
        ACL: 'public-read', // allow file to be read by anyone
        ContentType: 'image/png', // image header for browser to be able to render image
        CacheControl: 'max-age=31536000, public' // caching header for browser
    };

    // Uploading files to the bucket
    try {
        const result = await s3.upload(params).promise();
        return result.Location;
    } catch (err) {
        console.log('upload error', err);
        throw err;
    }

How to Upload files to Amazon AWS3 with NodeJS?

2 Answers2