How to Save and Retrieve a pdf file in MongoDB in NodeJS

Question

I have an issue where I have saved a small pdf file (~128KB) that is created in my Node.js server backend using Express to a document in Mongodb. I didn't use the Mongo GridFS because the files will always be under the 16MB limit. The collection has a schema of the following:

const mongoose = require('mongoose');
const Schema = mongoose.Schema;

const ReportsSchema = new Schema({
  ReportFileData: Buffer,
  Cert_objid: { type: Schema.Types.ObjectId, ref: 'Certs' },
  Report_Type: String,
  Note: String,
  Dau_objID: { type: Schema.Types.ObjectId, ref: 'Dau' },
  Pau_objID: { type: Schema.Types.ObjectId, ref: 'Pau' }
});

module.exports = Reports = mongoose.model('Reports', ReportsSchema);

I use the following code to create the pdf file and save it. I didn't include the docDefinition as it's producing the correct file if it's saved directly to the server.

const PdfPrinter = require('pdfmake/src/printer');
const path = require('path');
const moment = require('moment');

const Reports = require('../../models/Reports');  // Mongoose Schema

const createFAReport = data => {
    docDefinition...

createPdfBinary(docDefinition, binary => {
  const rpt = new Reports({
    ReportFileData: binary,
    Cert_objid: data._id,
    Report_Type: 'Water Use Report',
    Note: 'Testing 123'
  });

  rpt.save(err => {
    if (err) throw err;
    });
});

const createPdfBinary = (pdfDoc, callback) => {
  const fonts = {
    Roboto: {
    normal: path.join(__dirname, '../../', '/fonts/Roboto-Regular.ttf'),
    bold: path.join(__dirname, '../../', '/fonts/Roboto-Medium.ttf'),
    italics: path.join(__dirname, '../../', '/fonts/Roboto-Italic.ttf'),
    bolditalics: path.join(__dirname, '../../', '/fonts/Roboto-MediumItalic.ttf')
    }
  };

  const printer = new PdfPrinter(fonts);
  const doc = printer.createPdfKitDocument(pdfDoc);
  const chunks = [];
  let result;

  doc.on('data', function(chunk) {
    chunks.push(chunk);
  });
  doc.on('end', function() {
    result = Buffer.concat(chunks);
    callback('data:application/pdf;base64,' + result.toString('base64'));
  });
  doc.end();
};

To then retrieve the encoded document from MongoDB and write it to a local file for testing I used the following code (please note that the aggregate is to get some associated fields required to retrieve the correct report):

router.get('/getReport', passport.authenticate('jwt', { session: false }), (req, res) => {
  Certs.aggregate([
    {
      $match: {
        Cert_ID: '1578'
      }
    },
    {
      $lookup: {
        from: 'reports',
        localField: '_id',
        foreignField: 'Cert_objid',
        as: 'rpt'
      }
    },
    {
      $unwind: {
        path: '$rpt',
        includeArrayIndex: '<<string>>',
        preserveNullAndEmptyArrays: false
      }
    }
  ]).then(result => {
    result.map(rslt => {
      console.log(rslt.Cert_ID);
      res.json({ msg: 'Got the report.' });

      const fullfilePath = path.join(__dirname, '../../', '/public/pdffiles/', `1578.pdf`
      );

      fs.writeFile(fullfilePath, rslt.rpt.ReportFileData, 'base64', () => {
        console.log('File Saved.');
       });
     });
   });
 });

Everything seems to work fine, except when I open the file I get an error that the file corrupted. I'm wondering if saving it to "base64" is a problem or is the datatype for MongoDB a problem. The data type is buffer so would you retrieve it as a buffer instead? Any help would be appreciated.

Save the name of pdf file and then retrieve it from MongoDB. You should have a folder for ex.: 'uploads/docs', then when you save a pdf file named ex: 'DOC1.pdf', you write the file to 'uploads/docs/DOC1.pdf' and you save the filename in MongoDB. I Hope I was clear enough. — Diamant, Jan 04 '19 at 00:48
Unfortunately, I don't understand your comment. Once I create the pdf file in the Node.JS function, I then want to save it to MongoDB and not to a local file. At a later time, my ultimate goal is to have it available to react to display on the screen, but until then I was testing by trying to output it from the database to a local server copy. — Heath, Jan 04 '19 at 16:56
@Heath, Diamant is saying that you should only store the name in your database. Don't actually store the file there. This is a quite common practice. You can store the name or a relative path in mongo and then put the actual file either on your local machine or a more scalable solution is to use AWS S3. Unless you are planning on streaming this data, you are most likely better off going with that as a convention. — unflores, Jan 05 '19 at 21:37
@unflores Thanks for the explanation, that was exactly what i wanted to say. — Diamant, Jan 07 '19 at 13:45
Thanks to both of you, I understand, I did that in previous projects but was thinking for the size of the documents and convenience I'd like to try an alternate of saving directly to the db. — Heath, Jan 07 '19 at 16:38

score 4 · Accepted Answer · answered Jan 08 '19 at 15:14

4

I wouldn't recomend writing pdfs or images directly to a database. Here is some info on why Storing Images in DB - Yea or Nay?

You typically save a filename and store the file on a filesystem, either your own or a more scalable option would be something like S3.

Here is a module that might help you https://www.npmjs.com/package/formidable and if you plan on rolling your own you can still get some inspiration from it.

answered Jan 08 '19 at 15:14

unflores

1,764
2
15
35

Thanks for the additional information, that is useful! – Heath Jan 08 '19 at 17:24
1

mongodb natively supports uploading files no need to use any 3rd parties anymore – Sheece Gardazi May 28 '20 at 17:10
@SheeceGardazi is correct. MongoDB has native support for storing very large files including pdfs: https://docs.mongodb.com/manual/core/gridfs/ – Yigit Alparslan Apr 25 '21 at 21:48

How to Save and Retrieve a pdf file in MongoDB in NodeJS

1 Answers1