33

I want to use node-mssql as a MSSQL database connector in a Node JS Express 4 web application. Route handler logic is handled in separate files.

How do I create a single/global connection pool and use it across several files where route logic is handled? I don't want to make a new connection pool in each route handler function/file.

Christiaan Westerbeek
  • 10,619
  • 13
  • 64
  • 89

6 Answers6

78

It's been 3 years since I asked and answered the question. Since then a few things have changed. Here's the new solution based on ES6, mssql 4 and Express 4 that I would suggest today.

Two key elements are at play here.

  1. Modules are cached after the first time they are loaded. This means that every call to require('./db') will return exactly the same object. The first require of db.js will run that file and create the promise and export it. The second require of db.js will return THAT same promise without running the file. And it's that promise that will resolve with the pool.
  2. A promise can be thenified again. And if it resolved before, it will immediately resolve again with whatever it resolved with the first time, which is the pool.

In server.js

const express = require('express')
// require route handlers.
// they will all include the same connection pool
const set1Router = require('./routes/set1')
const set2Router = require('./routes/set2')

// generic express stuff
const app = express()

// ...
app.use('/set1', set1Router)
app.use('/set2', set2Router)

// No need to connect the pool
// Just start the web server

const server = app.listen(process.env.PORT || 3000, () => {
  const host = server.address().address
  const port = server.address().port

  console.log(`Example app listening at http://${host}:${port}`)
})

In db.js

const sql = require('mssql')
const config = {/*...*/}

const poolPromise = new sql.ConnectionPool(config)
  .connect()
  .then(pool => {
    console.log('Connected to MSSQL')
    return pool
  })
  .catch(err => console.log('Database Connection Failed! Bad Config: ', err))

module.exports = {
  sql, poolPromise
}

In routes/set1.js and routes/set2.js

const express = require('express')
const router = express.Router()
const { poolPromise } = require('./db')

router.get('/', async (req, res) => {
  try {
    const pool = await poolPromise
    const result = await pool.request()
        .input('input_parameter', sql.Int, req.query.input_parameter)
        .query('select * from mytable where id = @input_parameter')      

    res.json(result.recordset)
  } catch (err) {
    res.status(500)
    res.send(err.message)
  }
})

module.exports = router

To summarize

You'll always get the same promise due to module caching and that promise will, again and again, resolve with the pool it resolved with the first time. Thus each router file uses the same pool.

BTW: there are easier ways to go about the try catch in the express route that I won't cover in this answer. Read about it here: https://medium.com/@Abazhenov/using-async-await-in-express-with-node-8-b8af872c0016

The old solution

This is the solution I posted 3 years ago, because I believed I had an answer that was worth to share and I couldn't find a documented solution elsewhere. Also in a few issues (#118, #164, #165) at node-mssql this topic is discussed.

In server.js

var express = require('express');
var sql     = require('mssql');
var config  = {/*...*/};
//instantiate a connection pool
var cp      = new sql.Connection(config); //cp = connection pool
//require route handlers and use the same connection pool everywhere
var set1    = require('./routes/set1')(cp);
var set2    = require('./routes/set2')(cp);

//generic express stuff
var app = express();

//...
app.get('/path1', set1.get);
app.get('/path2', set2.get);

//connect the pool and start the web server when done
cp.connect().then(function() {
  console.log('Connection pool open for duty');

  var server = app.listen(3000, function () {

    var host = server.address().address;
    var port = server.address().port;

    console.log('Example app listening at http://%s:%s', host, port);

  });
}).catch(function(err) {
  console.error('Error creating connection pool', err);
});

In routes/set1.js

var sql     = require('mssql');

module.exports = function(cp) {
  var me = {
    get: function(req, res, next) {
      var request = new sql.Request(cp);
      request.query('select * from test', function(err, recordset) {
        if (err) {
          console.error(err);
          res.status(500).send(err.message);
          return;
        }
        res.status(200).json(recordset);
      });
    }
  };

  return me;
};
Christiaan Westerbeek
  • 10,619
  • 13
  • 64
  • 89
  • 1
    What are you supposed to do if the connection pool closes or has an error? If it goes down and you restart it, how can you update the connection across all of the other routes files? – Wait what Nov 17 '16 at 02:09
  • I followed most of your code but omitted passing in `cp`. And in set1.js I have no parameter in the `Request` call. And it still worked! I am worried about the implication of this. Is there still connection pooling at work with this method? Where did the code in set1.js get the connection from? – Old Geezer Feb 15 '17 at 15:03
  • 1
    @OldGeezer According to the docs, there's a global pool used in this case Quote : `If you omit pool/transaction argument, global pool is used instead.` ref: (https://github.com/patriksimek/node-mssql#request) – snajahi May 30 '17 at 11:01
  • In the latest version the following Line var cp = new sql.Connection(config); has changed to var cp = new sql.ConnectionPool(config); – ajaysinghdav10d Nov 21 '17 at 12:09
  • I am getting ResourceTimeoutError occasionally. What to do in this case of error? I have added to listen to pool's error event and reconnect the pool after closing the pool. Let me know if it is wrong. Thanks! – bugwheels94 Mar 19 '18 at 23:00
  • I updated my answer to reflect new insights, current standards and also incorporates suggestions from other answers. I would be glad to receive comments for improvements. – Christiaan Westerbeek Jun 15 '18 at 12:40
  • Many thanks, this solution works fine, after earlier I had many issues related to handling the connection. – Jay Jul 30 '18 at 15:16
  • Exporting `module.exports = { sql, poolPromise }` and importing and using it this way: `const { poolPromise } = require('./db') `, `const pool = await poolPromise` will not work. Should be `const pool = await poolPromise.poolPromise` or better naming the variables. – Maihan Nijat Oct 05 '18 at 15:30
  • @MaihanNijat I don't think that's correct. `const pool = await poolPromise.poolPromise` leads to pool -> undefined – Christiaan Westerbeek Jan 03 '19 at 10:17
  • @ChristiaanWesterbeek "And it's that promise that will resolve with the pool.".. I think the answer should have said "it's that promise that will resolve with a Connection from the pool"; Which leads me to something missing from your answer - after each query, the connection should be released back to the pool - using the right terminology e.g. dbConn instead of pool, made me aware dbConn.release() is missing; [But great answer anyway] – joedotnot Apr 11 '21 at 18:01
6

When you configure your app (like when you create the express server), make the DB connection. Make sure this is done BEFORE you require all your routes! (finagle the requires at the top of the file)

Just like the docs:

var sql = require('mssql'); var connection = new sql.Connection(..... //store the connection sql.globalConnection = connection;

Then in all your route files, you can do this:

var sql = require('mssql'); var sqlConn = sql.globalConnection; var request = new sql.Request(sqlConn); //...

That should do it!

All that said, go use knex to manage your MySQL query building. It has a built in connection pool, and you store the connected knex instance the same way. As well as a generous helping of awesome.

clay
  • 5,917
  • 2
  • 23
  • 21
  • 1
    This statement: sql.globalConnection = connection; Is that perfectly fine for a web server? I mean is there going to be problems with many users? – Amiga500 Dec 17 '15 at 14:42
  • My example was just a way to save this connection variable across files. Takes advantage of the way `require()` works in Node (basically a global, shared object, and you can add something to it). A connection pool is a better idea for a web server to handle more concurrent DB actions. – clay Dec 17 '15 at 18:51
  • Can you provide me an example how to load it using connection pool? – Amiga500 Dec 18 '15 at 08:00
  • 1
    In my answer, I'm advocating using `knexjs`. Here is [an example](http://knexjs.org/#Installation-pooling) config for `knexjs`. Once connected, you do not have to do anything with the connection pool. Looks like you can [just use the pool](https://github.com/felixge/node-mysql#pooling-connections) in `node-mysql` as well. – clay Dec 18 '15 at 13:25
5

src/config.js

export default {
  database: {
    server: process.env.DATABASE_SERVER || '<server>.database.windows.net',
    port: 1433,
    user: process.env.DATABASE_USER || '<user>@<server>',
    password: process.env.DATABASE_PASSWORD || '<password>',
    database: process.env.DATABASE_NAME || '<database>',
    connectionTimeout: 30000,
    driver: 'tedious',
    stream: false,
    options: {
      appName: '<app-name>',
      encrypt: true
    }
  }
};

src/server.js

import sql from 'mssql';
import express from 'express';
import config from './config';

// Create and configure an HTTP server
const server = express();
server.set('port', (process.env.PORT || 5000));

// Register Express routes / middleware
server.use('/api/user', require('./api/user');

// Open a SQL Database connection and put it into the global
// connection pool, then launch the HTTP server
sql.connect(config.database, err => {
  if (err) {
    console.log('Failed to open a SQL Database connection.', err.stack);
  }
  server.listen(server.get('port'), () => {
    console.log('Node app is running at http://127.0.0.1:' + server.get('port'));
  });
});

sql.on('error', err => console.log(err.stack));

src/api/user.js

import sql from 'mssql';
import { Router } from 'express';

const router = new Router();

router.get('/:id', async (req, res, next) => {
  try {
    const request = new sql.Request();
    request.input('UserID', req.params.id);
    request.multiple = true;

    const dataset = await request.query(`
      SELECT UserID, Name, Email
      FROM [User] WHERE UserID = @UserID;
      SELECT r.RoleName FROM UserRole AS r
        INNER JOIN [User] AS u ON u.UserID = r.UserID
      WHERE u.UserID = @UserID
    `);

    const user = dataset[0].map(row => ({
      id: row.UserID,
      name: row.Name,
      email: row.Email,
      roles: dataset[1].map(role => role.RoleName)
    })).shift();

    if (user) {
      res.send(user);
    } else {
      res.statusCode(404);
    }
  } catch (err) {
    next(err);
  }
});

export default router;

See also MSSQL SDK for Node.js, T-SQL Reference, React Starter Kit

Konstantin Tarkus
  • 37,618
  • 14
  • 135
  • 121
  • 1
    this example uses typescript, replace `import sql from 'mssql';` etc. with `const sql = requires('mssql');`. please note that `const` won't work if you're using an older version of node, use `var` instead. also `export default ` should be replaced with `module.exports = `. – snajahi May 30 '17 at 10:44
5

This is how I did it which I think is a little simpler than some of the other solutions.

Database File (db.js):

const sql = require('mssql')

const config = {}

const pool = new sql.ConnectionPool(config)
  .connect()
  .then(pool => {
    console.log('Connected to MSSQL')
    return pool
  })
  .catch(err => console.log('Database Connection Failed! Bad Config: ', err))

module.exports = {
  sql, pool
}

Query:

const { pool, sql } = require('../db')

return pool.then(conn => {
    const ps = new sql.PreparedStatement(conn)
    ps.input('xxxx', sql.VarChar)

    return ps.prepare(`SELECT * from table where xxxx = @xxxx`)
      .then(data => ps.execute({ xxxx: 'xxxx' }))
  })

EDIT: Updated to match Christiaan Westerbeek's gist which was much cleaner.

ozzieisaacs
  • 833
  • 2
  • 11
  • 23
  • Today, I would probably do something like this too. But I don't think you need the timeout. Also, a promise that already resolved earlier can be thenified again in which case it will resolve with the earlier result immediately. So, you don't need the timeout nor the condition. – Christiaan Westerbeek Jun 13 '18 at 21:19
  • If you don't have the timeout and try to run a query before the pool is ready it will still be null. This way we just keep waiting 200ms each time until the pool is finally initialized. Maybe I am not understanding your explanation? – ozzieisaacs Jun 13 '18 at 21:53
  • 1
    Thanks for the gist, that's a much better way to do it! I ignored that it returned a promise already. I'm going to edit what I wrote above. – ozzieisaacs Jun 14 '18 at 16:19
  • I removed the link to my gist since it's incorporated in my updated answer – Christiaan Westerbeek Jun 24 '18 at 07:23
  • why no use of the Unprepare after the statement? – platinums Mar 27 '19 at 17:40
4

I used similar concept (single connection pool), but wrapped the connection logic in one file (No need to pass connection pool to other places). The connPoolPromise below will only be initialized once since modules are cached after the first time they are loaded.

e.g. DBUtil.js

const sql = require('mssql');
const dbConfig = require('./dbconfig');
let connPoolPromise = null;

const getConnPoolPromise = () => {
  if (connPoolPromise) return connPoolPromise;

  connPoolPromise = new Promise((resolve, reject) => {
    const conn = new sql.ConnectionPool(dbConfig);

    conn.on('close', () => {
      connPoolPromise = null;
    });

    conn.connect().then(connPool => {
      return resolve(connPool);
    }).catch(err => {
      connPoolPromise = null;
      return reject(err);
    });
  });

  return connPoolPromise;
}

// Fetch data example using callback
exports.query = (sqlQuery, callback) => {

  getConnPoolPromise().then(connPool => {

    return connPool.request().query(sqlQuery);

  }).then(result => {
    callback(null, result);
  }).catch(err => {
    callback(err);
  });

};

Usage user.js:

const DBUtil = require('./DBUtil');
DBUtil.query('select * from user where userId = 12', (err, recordsets) => {
  if (err) return callback(err);

  // Handle recordsets logic

}
Jonathan
  • 51
  • 4
  • 1
    i like this idea, to get it working now var conn = new sql.Connection has to be change to var conn = new sql.ConnectionPool – platinums Mar 27 '19 at 12:39
  • I like this one, for me I had to change the query function. the callback I had to put in with the query. Like this: connPool.request().query(sqlQuery, callback) and remove the then and catch in there. – Andres Sep 03 '19 at 16:26
  • Thanks for this. I was testing the original answer with making the SQL connection go down and it didn't handle it well. The way you return the promise and set it to null on errors works well :) – Adam91Holt Dec 13 '19 at 02:31
0

Not crazy about the examples I've seen so far for setting up pooled connection. I do:

const pool = new mssql.ConnectionPool(msConfig).connect()
  .then(_ => { return _ } )
  .catch(e => console.error("Database Trouble!  ", e))
  
 /* ... */
 
 pool
  .then(_ => _.query( /* ... */ )
  .then(result => { /* ... */ })
  .catch(e => { /* ... */ })
  • me either, the npm pages are all over the place also. You have by far the most succinct version of a request ive seen, Well Done! – platinums Nov 26 '19 at 16:47