4

I want to create a cluster of node.js server in order to support high concurrency, for a chat rooms application. I need to be able to share information between all nodes. I am trying to find out what would be the best way to keep all the servers in-sync. I want as much flexibility as possible in the shared object, as I plan to add more features in the future.

So far, I have 2 solutions in mind:

  1. Subscribe to NoSQL key (for example redis publish-subscribe)
  2. Nodes update each other using sockets.

Which is better? Any other ideas?

Kuf
  • 17,318
  • 6
  • 67
  • 91
  • "Keeping all clusters synched" is a huge discussion, I guess you need an eventually consistency model (http://en.wikipedia.org/wiki/Eventual_consistency). Redis is a reliable solution as both PUB/SUB & storage (considering a more persistent one would also be a good idea). For IPC you could even try nodejs native IPC (http://nodejs.org/api/cluster.html#cluster_worker_send_message_sendhandle). Using sockets would be a painful idea as you need to implement your own "protocol". – Evan P Jun 06 '13 at 10:03
  • @Evalon The link to [node.js clusters](http://nodejs.org/api/cluster.html#cluster_worker_send_message_sendhandle) is for single server cluster, not for multi-server, that's why i though about using sockets to communicate between different servers. This is my first node project, and I wish to know which is better, the socket sync or the redis-subscribe. – Kuf Jun 06 '13 at 10:16
  • So you want distributed clusters not just a fork. I see, then message brokers should also be on the table. http://www.zeromq.org/bindings:node-js , https://github.com/postwait/node-amqp . If you insist an implementation using sockets, you should read as well http://stackoverflow.com/questions/10557826/node-js-socket-io-redis-pub-sub-high-volume-low-latency-difficulties – Evan P Jun 06 '13 at 10:51

1 Answers1

5

Redis is nice because it's independent of your node app and fairly easy to scale. You can also use it for a lot of stuff outside of pub/sub as well, such as sharing basic data structures (hashes, sorted sets, lists, strings) between your node servers to help keep them in sync this way as well. Theoretically, you could save all chats in a given room as a sorted set where your key is a json representation of some chat object (something like {'user':'some_user','msg':'some_msg'} and your score is the timestamp, so it's very easy to pull conversations by time). Redis is extremely fast, and its data structures are highly optimized, so a single server can handle many, many users.

We have a similar setup in production with one Redis server handling 1 million users (about 10k hits inserts and 20k reads from a sorted set per minute), and the CPU usage rarely gets above 5% on a non-CPU-heavy box.

Eli
  • 36,793
  • 40
  • 144
  • 207
  • Eli can you tell me more "We have a similar setup in production with one Redis server handling 1 million users (about 10k hits inserts and 20k reads from a sorted set per minute)," what kinda server ? how many connection (websocket?) ? thanks – LXG Jul 14 '13 at 12:49
  • @LXG server was an m2.xlarge. There were usually a pretty small but variable number of application servers connected-- in the range of 20. – Eli Sep 13 '13 at 18:42
  • Do you really mean "per minute"? 20k / 60 = ~333 requests per second? – scaryguy Jul 03 '15 at 11:52