1

I am just learning how to use node/express and mongodb, and I am a little confused about pooling connections works.

My strategy right now is to have the database connections at the router level, so each route has its own connection. My code for one of my routes looks like this:

var express = require('express');
var router = express.Router();

var config = require('../config.json');

var url = config.db.URI;

var MongoClient = require('mongodb').MongoClient
    , assert = require('assert');

var db;

// Connect to database
router.all('*', function(req, res, next) {
    console.log('Connecting to db');
    MongoClient.connect(url, function(err, database) {
        assert.equal(null, err);
        db = database;
    });
    next();
});

// GET admin list page
router.get('/', function(req, res, next) {
    res.render('lists/index');
    var coll = db.collection('lists');
    coll.find().each(function(err, obj) {
        console.log(obj);
    });
    next();
});

router.get('/new', function(req, res, next) {
    res.render('lists/new');
    next();
});

router.all('*', function(req, res, next) {
    console.log('Closing database');
    //db.close();
});

module.exports = router;

I am testing how this view interacts with my database with the two router.all functions wrapping my http requests. In my mind, any time a page in this router is requested, the server should connect to the database, allow the corresponding http request function to access the database (in this case just printing its contents to console), then close it at then end once the http request is done. However, this is not the case. As you can see I've commented out the db.close function as it throws an error saying "Can't set headers once they are sent" with it. I assume this means that I don't quite understand the Express route "pipeline" and things are not executing in the order I expect them too.

Is there a different router method I should be using? Maybe router.use? Or am I approaching this in the wrong way, should I put my database connection somewhere else?

I am using this as a reference somewhat on what is good practice as far as pooling goes, but they don't seem to close the connection? If I don't close the connection I just see my number of mongodb connections go up and up, which I don't think is a good thing. Is it just that you have to find a balance between the number and length of connections?

Geoff McLennan
  • 418
  • 1
  • 6
  • 15

1 Answers1

2

You should make your connection once and use it in all handlers.

Right now what you're doing is reconnecting to the database on every single request, even on those requests that don't use the database at all. Besides, you're calling next() before even waiting for the connection to get established.

Database connections are meant to be persistent - not one-time things, so you're likely to get very poor performance with your approach and I'm not even sure why would you want to do that. Did you have any problems with a single connection? I don't think that hammering your database like that would help. If anything, it can only make things worse.

When you connect to Mongo with the native MongoDB Node.js Driver there are some options that you can use, like:

  • poolSize - Set the maximum poolSize for each individual server or proxy connection (default is 5)
  • autoReconnect - Reconnect on error (default is true)

Some other interesting options are: reconnectTries, reconnectInterval, keepAlive, connectTimeoutMS, socketTimeoutMS.

You can change the values of those options if you're not happy with the defaults.

For more info see:

rsp
  • 107,747
  • 29
  • 201
  • 177
  • So if I should connect to the database once on startup of the website server, then use that connection for all queries as needed? Will that just be one connection to the database? Or will each user that accesses the site start a new connection? I do agree that I don't want to hammer the database with connection requests, but I did not think it was a good idea to leave a connection open for the lifetime of the program so I was trying to find some sort of balance. This is my first time using a database like mongo, so I'm really not sure what the best practices are. – Geoff McLennan Nov 30 '16 at 20:06
  • 1
    Would an approach like [this](https://www.terlici.com/2015/04/03/mongodb-node-express.html) be more appropriate? – Geoff McLennan Nov 30 '16 at 20:11
  • @GeoffMcLennan Yes, it's fine to open connection once and just use it. You can change the options that configure reconnecting, pooling etc. if you're not happy with the defaults. See the links that I added to the answer. – rsp Nov 30 '16 at 21:14