1

I am building a NodeJS Express API that accepts a filepath as a parameter, gets an object from S3 and then sends the file back in the API response.

I am using the NodeJS Amazon AWS S3 SDK @aws-sdk/client-s3.

I have everything working up to returning the file in the API response.

Here is the controller. This is working to get the object as a readable stream from S3.

const consoleLogger = require('../logger/logger.js').console;
const { S3Client, GetObjectCommand } = require('@aws-sdk/client-s3');
const s3client = new S3Client({ region: process.env.AWS_REGION || 'us-east-1' });

const awsCtrl = {};

awsCtrl.getObject = async (key) => {

  // Get object from Amazon S3 bucket
  let data;
  try {
    // Data is returned as a ReadableStream
    data = await s3client.send(new GetObjectCommand({ Bucket: process.env.AWS_BUCKET, Key: key }));
    console.log("Success", data);
  } catch (e) {
    consoleLogger.error("AWS S3 error: ", e);
    const awsS3Error = {
      name: e.name || null,
      status: e.$metadata.httpStatusCode || 500
    };
    throw awsS3Error;
  }

  return data;

}

module.exports = awsCtrl;

Here is where the controller is called.

router.get('/:fileId', (req, res, next) => {

  return filesCtrl.deliverFile(req, res);

});

filesCtrl.deliverFile = async (req, res) => {
  /* Get object from AWS S3 */
  let fileStream;
  try {
    fileStream = await awsCtrl.getObject(filePath);
  } catch (e) {
    consoleLogger.error(`Unable to get object from AWS S3`, e);
    if (e.status && e.status === 404) {
      result.error = `Not found`;
      result.status = 404;
      return res.status(result.status).json(result);
    }
    return res.status(e.status || 500).json(result);
  }

  // Set file header
  res.attachment(filename);

  // HOW TO RETURN THE FILE IN API RESPONSE TO DOWNLOAD?
  // not working
  //return fileStream.pipe(res); 
}

Question:

How do I return the file in the Node JS Express API response to download?

Note: I've already tried other suggestions in a few other questions but none of the solutions have worked to return the file in the API response.

I tried the answer in this question here but the file is downloaded on the server side, the file needs to be returned in the API response. How to save file from S3 using aws-sdk v3

Also the answer in this question here is using the v2 version of the SDK and this is v3. There is no .createReadStream() method anymore. NodeJS download file from AWS S3 Bucket

pengz
  • 2,279
  • 3
  • 48
  • 91
  • 2
    I might be missing something... what does "API response" mean in your context? Is this express? I'm not familiar with v3 of the sdk, but v2 would return a buffer... are you asking how to return binary data? – 404 Sep 18 '21 at 16:25
  • @404 Yes this is a NodeJS Express API. I want to return the file in the Express API response 'res' to download the file on the client side. – pengz Sep 19 '21 at 00:51
  • 1
    You really want to sent it as a file for sure ? **Or** is it ok if you send a download a link to client, which can be used by client to download the file in FE ? – Abiram Sep 19 '21 at 03:20
  • 3
    If you are using lambda don't do this, use S3 presigned urls! You will pay for download time from S3 to lambda, and download time from lambda to end user. – Lucasz Sep 19 '21 at 07:35
  • @Abiram yes it needs to be sent as a file to download on the client side. – pengz Sep 21 '21 at 14:40
  • @Lucasz Thank you we are not using lambda this is a standalone NodeJS application on our own servers. – pengz Sep 21 '21 at 14:41
  • 1
    `ReadableStream` is part of Web Streams API (as implemented in `node:stream/web`). I can't confirm that it is what s3 sdk really uses, but if that is the case, `pipe()` is not supposed to be defined. Instead, it should be `pipeTo()`. But I'm afraid you're going to have to convert web standard stream to node `Readable` stream. – Yuri Sh Sep 22 '21 at 06:11

1 Answers1

2

Server Side

const aws = require('aws-sdk');

router.get('/getfilefroms3', async (req, res, next) => {

  aws.config.update({
    secretAccessKey: config.secret_access_key,
    accessKeyId: config.access_key_id,
    signatureVersion: config.signature_version,
    region: config.region
  })

  const s3 = new aws.S3({ });

  var params = { Bucket: config.sample_bucket_name, Key: req.query.filename };

  s3.getObject(params, function (err, data) {
    if (err) {
      res.status(200);
      res.end('Error Fetching File');
    }
    else {
      res.attachment(params.Key); // Set Filename
      res.type(data.ContentType); // Set FileType
      res.send(data.Body);        // Send File Buffer
    }
  });

})

Client Side

If you are using Web Application you can use any HTTP REST API Client like Axios or Fetch, The Download Manager will capture the file.

curl --location --request GET 'http://localhost:5001/getfilefroms3?filename=sample.pdf'

If you are using NodeJS Application

var http = require('http');
var fs = require('fs');

var download = function (url, destination, callback) {
    var file = fs.createWriteStream(destination);
    http.get(url, function (response) {
        response.pipe(file);
        file.on('finish', function () {
            file.close(callback);
        });
    });
}

var fileToDownload = "sample.pdf"

download("http://localhost:5001/getfilefroms3?filename=" + fileToDownload, "./" + fileToDownload, () => { console.log("File Downloaded") })
Abiram
  • 188
  • 1
  • 1
  • 7