Upload pdf generated to AWS S3 using nodejs aws sdk

Question

I am using pdfkit to generate a pdf with some custom content and then sending it to an AWS S3 bucket.

While if I generate the file as a whole and upload it works perfectly, however, if I want to stream the generated file probably as an octet stream I am not able to find any relevant pointers.

I am looking for a nodejs solution (or suggestion).

Have you tried using ```request``` module? You should be able to pipe ```doc``` to ```request```. — edin-m, Dec 14 '15 at 10:34
Okay I have found out the solution. The way to go about doing so , is use upload instead of putObject with the s3 api. And thus you can use a readable stream as body of the s3 upload params. — Shivendra Soni, Dec 14 '15 at 15:30
Welcome to stackoverflow. Post an answer since someone may find it usefull. — edin-m, Dec 14 '15 at 15:38
Added it as an answer. Please tell if there is something more I should add. — Shivendra Soni, Dec 15 '15 at 19:57

score 31 · Accepted Answer · edited May 31 '17 at 16:42

31

I'll try to be precise here. I will not be covering usage of pdfKit's nodejs sdk in much detail.

IF you want your generated pdf as a file.

var PDFDocument = require('pdfkit');

// Create a document
doc = new PDFDocument();

// Pipe it's output somewhere, like to a file or HTTP response
doc.pipe(fs.createWriteStream('output.pdf'));
doc.text('Whatever content goes here');
doc.end();
var params = {
  key : fileName,
  body : './output.pdf',
  bucket : 'bucketName',
  contentType : 'application/pdf'
}

s3.putObject(params, function(err, response) {

});

However if you want to stream it ( to say S3 bucket in the context of question), then it is worth remembering that every pdfkit instance is a readable stream.

And S3 expects a file, a buffer or a readable stream. So,

var doc = new PDFDocument();

// Pipe it's output somewhere, like to a file or HTTP response
doc.text("Text for your PDF");
doc.end();

var params = {
  key : fileName,
  body : doc,
  bucket : 'bucketName',
  contentType : 'application/pdf'
}

//notice use of the upload function, not the putObject function
s3.upload(params, function(err, response) {

});

edited May 31 '17 at 16:42

Anthony F.

545
1
5
20

answered Dec 15 '15 at 19:56

Shivendra Soni

672
2
6
19

Are you using `aws-sdk` or another module? Trying to accomplish this now with `knox`, using `putStream`. My file is not being saved on the file system, only streamed to s3. – Nigel Earle Feb 19 '16 at 00:29
Yes, I am using the aws-sdk – Shivendra Soni Feb 29 '16 at 18:15
1

'Cannot determine length of [object PDFDocument]': use s3.upload() instead: http://stackoverflow.com/questions/30227352/determining-the-length-of-a-read-stream-in-node-js – ricka Mar 23 '17 at 20:45
@ShivendraSoni: can we bind html content in doc.text()? – shiva May 07 '18 at 05:44
@shiva I don't think so. Although this is slightly out of context for the question. You should use pdfkit's apis which allow you to add text, images etc to the pdf. However if you want to bind html content only, one way I see you can do it is, convert it into a vector ( paint it on canvas tag) and then bind it. Hope it helps. – Shivendra Soni May 07 '18 at 08:28
@shiva Alternatively, a more direct approach would be to use. https://github.com/devongovett/node-wkhtmltopdf instead. It's tailor made for this specific use case and you won't have to manually adjust the contents of pdfkit (too low level I guess) – Shivendra Soni May 07 '18 at 08:30
@ShivendraSoni: Yes i have used html-pdf npm package to handle my problem. Btw Thanks for the info – shiva May 07 '18 at 09:01
1

writing to disk is not guaranteed to have finished by the time you start uploading. Using the above method can result to corrupt files. see https://github.com/foliojs/pdfkit/issues/265 – Ioannis Tsiokos Jul 02 '19 at 07:15

score 5 · Answer 2 · answered Sep 21 '20 at 13:04

5

If you are using html-pdf package and aws-sdk than it's very easy...

var pdf = require('html-pdf');
import aws from 'aws-sdk';
const s3 = new aws.S3();

pdf.create(html).toStream(function(err, stream){
  stream.pipe(fs.createWriteStream('foo.pdf'));
  const params = {
                Key: 'foo.pdf',
                Body: stream,
                Bucket: 'Bucket Name',
                ContentType: 'application/pdf',
            };
  s3.upload(params, (err, res) => {
                if (err) {
                    console.log(err, 'err');
                }
                console.log(res, 'res');
            });
});

answered Sep 21 '20 at 13:04

Subham kuswa

356
4
12

stream.pipe throws error `Error: ENOENT: no such file or directory, open './2020-2021/11761.pdf']` – Lalitesh Upadhyaya Mar 20 '21 at 12:33
`2020-2021` is this folder exist ? the main purpose of this code block is to generate pdf file from html and upload it to s3 directly. the file will not be stored in your directory. – Subham kuswa Jul 13 '21 at 12:20

score 4 · Answer 3 · answered Jun 24 '21 at 13:18

Tried this and worked. I created a readFileSync and then uploaded that to S3. I also used a "writeStream.on('finish' " so that pdf file is completely created before it is uploaded, otherwise it uploads partial file.

const PDFDocument = require('pdfkit');
const fs = require('fs');
const AWS = require('aws-sdk');
const path = require('path')

async function createPDF() {


const doc = new PDFDocument({size: 'A4'});
let writeStream = fs.createWriteStream('./output.pdf')
doc.pipe(writeStream);


// Finalize PDF file
doc.end();

writeStream.on('finish', function () {
    var appDir = path.dirname(require.main.filename);
    const fileContent = fs.readFileSync(appDir + '/output.pdf');
    var params = {
        Key : 'filName',
        Body : fileContent,
        Bucket : process.env.AWS_BUCKET,
        ContentType : 'application/pdf',
        ACL: "public-read"
      }
      
    const s3 = new AWS.S3({
        accessKeyId: process.env.AWS_ACCESS_KEY,
        secretAccessKey: process.env.AWS_SECRET_KEY
    });
      //notice use of the upload function, not the putObject function
    s3.upload(params, function(err, response) {
        
    });
});

}

score 0 · Answer 4 · answered Nov 16 '22 at 21:56

This is how I uploaded a PDF file with Cypress to AWS S3:

cy.readFile("cypress/fixtures/Document.pdf", "base64").then((fileBase64) => {
    const fileBlob = Cypress.Blob.base64StringToBlob(fileBase64, "application/pdf");
    uploadFileToS3(blob);
});

Upload pdf generated to AWS S3 using nodejs aws sdk

4 Answers4

Linked