NodeJS + SocketIO: Scaling and preventing single point of failure

Question

So the first app that people usually build with SocketIO and Node is usually a chatting app. This chatting app basically has 1 Node server that will broadcast to multiple clients. In the Node code, you would have something like.

//Psuedocode
for(client in clients){
  if(client != messageSender){
    user.send(message);
  }
}

This is great for a low number of users, but I see a problem with this. First of all, there is a single point of failure which is the Node server. Second of all, the app will slow down as the number of clients grow. What is there to do then when we reach this bottleneck? Is there an architecture (horizontal/vertical scaling) that can be used to alleviate this problem?

How many clients do you expect to be connected to the server? For that loop to be noticeably slow I would say you need several thousands, if not tens of thousands. — Some programmer dude, Mar 16 '12 at 06:27

score 4 · Accepted Answer · answered Mar 18 '12 at 07:14

For that "one day" when your chat app needs multiple, fault-tolerant node servers, and you want to use socket.io to cross communicate between the server and the client, there is a node.js module that fits the bill.

https://github.com/hookio/hook.io

It's basically an event emitting framework to cross communicate between multiple "things" -- such as multiple node servers.

It's relatively complicated to use, compared to most modules, which is understandable since this is a complex problem to solve.

That being said, you'd probably have to have a few thousand simultaneous users and lots of other problems before you begin to have problems with this.

Another thing you can do, is try to develop your application in a way so that if a connection is lost (which happens all the time anyway), eg. server goes down, client has network issues (eg. mobile user), etc, your application should be able to handle that and recover from such issues gracefully.

score 2 · Answer 2 · edited May 23 '17 at 11:56

Since Node.js has a single event-loop thread, this single point of failure is written into its DNA. Even reloading a server after code changes require this thread to be stopped.

There are however a lot of tools available to handle such failures gracefully. You could use forever; a simple CLI tool for ensuring that a given script runs continuously. Other options include distribute and up. Distribute is a load balancing middleware for Node. Up builds on top of Distribute to offer zero downtime reloads using either a JavaScript API or command line interface:

Further reading I find you just need to use Redis Store with Socket.io to maintain connection references between two or more processes/ servers. These options have already been discussed extensively here and here.

There's also the option of using socket.io-clusterhub if you don't intend to use the Redis store.

have you ever tried using distribute? I am just wondering how it would be able to broadcast to all users. it seems like each node has to know about all users that are connected. — denniss, Mar 16 '12 at 07:45

NodeJS + SocketIO: Scaling and preventing single point of failure

2 Answers2