How to instantiate Multiple Redis Connections for Publish Subscribe (node.js + node_redis)

Question

Scenario

Using node_redis to build a simple Redis Pubish Subscribe (chat) example: https://github.com/nelsonic/hapi-socketio-redis-chat-example (with Hapi.js and Socket.io)

We have created a node module redis_connection.js in our project ( see: http://git.io/vqaos ) to instantiate the Redis connection because we don't want to be repeating the code which connects (to RedisCloud) multiple times:

var redis = require('redis');
var url   = require('url');
var redisURL    = url.parse(process.env.REDISCLOUD_URL);
var redisClient = redis.createClient(redisURL.port, redisURL.hostname,
                  {no_ready_check: true});
redisClient.auth(redisURL.auth.split(":")[1]);

module.exports = redisClient;

Which we then use like this:

var redisClient = require('./redis_connection.js');

// Confirm we are able to connect to  RedisCloud:
redisClient.set('redis', 'working', redisClient.print);
redisClient.get('redis', function (err, reply) {
  console.log('RedisCLOUD is ' +reply.toString());
});

This works fine for normal GET/SET operations with Redis, but when we try to instantiate multiple connections to Redis (e.g: one to publish, another to subscribe and a third just to GET/SET keys/values) we get an error:

Issue

We are seeing the following error:

Error: Connection in subscriber mode, only subscriber commands may be used

What are we doing wrong?

Full code at the point where we see this issue: http://git.io/vqa6y

Note

We tried to dig through existing SO Q/A on this, e.g:

but did not find a solution that exactly matched our situation...

(any suggestions/help much appreciated!)

Did any of the suggested solutions worked out for you? – Ofir Luzon Jul 12 '15 at 15:56 — Ofir Luzon, Jul 12 '15 at 15:56

score 3 · Answer 1 · answered Jul 09 '15 at 06:21

3

Not tested, but too long for a comment.

Try to define another redis connection module, one for your regular usage and a second one solely for your pubsub subscriptions usage.

Add a redis_pubsub_connection.js to your project:

var redis = require('redis');
var url   = require('url');
var redisURL    = url.parse(process.env.REDISCLOUD_URL);
var redisPubSubClient = redis.createClient(redisURL.port, redisURL.hostname,
                  {no_ready_check: true});
redisPubSubClient.auth(redisURL.auth.split(":")[1]);

module.exports = redisPubSubClient;

And change your publish.js require statement to:

var redis = require('./redis_pubsub_connection'); // RedisCloud

answered Jul 09 '15 at 06:21

Ofir Luzon

10,635
3
41
52

This is a simple solution to the problem but would require 3 very similar modules: **redis_pub_connection.js** + **redis_sub_connection.js** + **redis_connection.js** because pub and sub cannot be the same and we still want to be able to GET/SET keys so we'd need the third one too. – nelsonic Jul 09 '15 at 08:12
@nelsonic, you should be good publishing messages on the regular connection. One separate connection for subscribers is needed. – Ofir Luzon Jul 09 '15 at 08:28

score 3 · Accepted Answer · edited Nov 21 '21 at 15:45

redis-connection node.js module

In the interest of keeping this re-useable across our projects we wrote a (mini) node.js module to initialize Redis connections: https://github.com/dwyl/redis-connection

The code is simple and tested and takes care of authentication if required. (not copy-pasting the module here to avoid duplication)
see: https://github.com/dwyl/redis-connection/blob/master/index.js

Usage:

Install from NPM

npm install redis-connection --save

Use in your script

var redisClient = require('redis-connection')();
redisClient.set('hello', 'world');
redisClient.get('hello', function (err, reply) {
  console.log('hello', reply.toString()); // hello world
});

Publish Subscribe

var redisClient = require('redis-connection')(); // Publisher
var redisSub = require('redis-connection')('subscriber');
redisSub.subscribe("chat:messages:latest", "chat:people:new");

For a working example see: https://github.com/dwyl/hapi-socketio-redis-chat-example

The advantage is that we can re-use the same redisClient across multiple files in the same project without creating new connections (the single or pub/sub connection is cached and re-used)

Credit: We borrowed ideas from several places so have up-voted all the answers. But ultimately we wrote a slightly different solution so we have shared it with everyone on NPM/GitHub. Thanks again everyone!

@AlokRajasukumaran StackOverflow is probably not the best place to ask these kinds of questions, but yes, we are still using it in production for a few of our node.js apps. If you need any support or want to update anything, please open an issue on GitHub. Thanks. — nelsonic, Dec 24 '21 at 09:49

Robert Moskal · Answer 3 · 2015-09-15T15:06:27.293

1

The problem is that your redis client creation code is being cached by requires so you reuse the same connection again and again. Instead of returning the connection in your redis_connection module, you could return a function:

module.exports = function(){
    var redis = require('redis'); 
    var url = require('url'); var redisURL = url.parse(process.env.REDISCLOUD_URL); 
    var redisClient = redis.createClient(redisURL.port, redisURL.hostname, {no_ready_check: true});
    redisClient.auth(redisURL.auth.split(":")
    return redisClient;
}

And then call it like so:

var redisClient = require('./redis_connection.js')();

edited Sep 15 '15 at 15:06

answered Jul 08 '15 at 11:09

Robert Moskal

21,737
8
62
86

This solution will open and close connection per request. – Ofir Luzon Jul 09 '15 at 06:22
@OfirLuzon we do *want* to open *multiple distinct* Redis connections but ***not*** for every request ... do you have an alternative suggestion? – nelsonic Jul 09 '15 at 08:03
My solution will open a distinct connection every time it's called. You can call it once in a module, when that module loads and then reuse that same connection throughout a modules lifetime. – Robert Moskal Jul 09 '15 at 10:57

Robert Moskal · Answer 4 · 2015-07-09T13:24:58.180

If you want to supply regular connection and a sub one and you want to ensure you only have one of each across the application than you could use a combination of the two solutions that includes the notion of a singleton, something like this:

var subConnection, con;

var createConnection = module.exports.createConnection = function(){
    var redis = require('redis'); 
    var url = require('url'); var redisURL = url.parse(process.env.REDISCLOUD_URL); 
    var redisClient = redis.createClient(redisURL.port, redisURL.hostname, {no_ready_check: true});
    redisClient.auth(redisURL.auth.split(":")
    return redisClient;
}

module.exports.getSubConnection = function(){

if (!subConnection) 
    subConnection = createConnection();

return subConnection
}

module.exports.getConnection = function(){

if (!con) 
    con = createConnection();

return con
}

}

Repeat for the oher two connection types and call it like

var con =  require('./redis_connection.js').getConnection();

A dedicated connection for publish is not needed. Messages can be published on the regular connection. — Ofir Luzon, Jul 09 '15 at 12:56