0

I'm thinking about writing a few web applications having almost the same requirements as a chat. And I would like them to be able to scale easily.

I have worked a bit with node.js and I understand how it can help design push applications but I have some difficulties when thinking about having them run on multiple servers.

Here are some design I can think of for a large scale chat app :

1 - Servers have state, they keep the connections opened and clients can have new messages pushed to them. In this scenario, we are limited by the physical memory of one server so we cannot scale linearly if we have too many users per room.

2 - Servers have no state, they request a distributed database to respond to clients requests. In this scenario, clients poll the servers. We could scale linearly but the throughput is decreased, the messages are not delivered instantly and polling has been shown as a bad practice when scaling.

3 - Mix of 1 and 2. Servers keep the connections of its clients opened and poll the distributed database. The application is more complex to write and we still use polling. Similar client's requests (clients of the same room) are just grouped into a single one done by the server. The code becomes unnecessary complicated and it does not scale in the situation where we have many rooms and a few users per room.

4 - Servers have no state and the database cluster uses event to notify every registered servers about new messages. This is the solution I would like to have but I haven't heard of any database which has this feature. (Some people are talking about this feature for mongodb here: https://jira.mongodb.org/browse/SERVER-124)

So Why is the 4th solution not used so much today?

How do people usually design their applications in this case?

Community
  • 1
  • 1
Tug
  • 189
  • 10

2 Answers2

4

Since you want a push application, you would probably use Socket.IO with RedisStore.

By using this combination, the data for all the connections is kept in Redis (in-memory database), so you can scale outside a process. Another use of Redis here is for pub-sub.

The idea is to trigger an event when something needs to be pushed, then sent a message to the browser using Socket.io. If you want to listen to database changes, perhaps it's better to use CouchDB with it's _changes feature.

Resources:

https://github.com/dshaw/talks/tree/master/2011-10-jsclub/sample-app
http://www.ranu.com.ar/2011/11/redisstore-and-rooms-with-socketio.html
How to reuse redis connection in socket.io?

Community
  • 1
  • 1
alessioalex
  • 62,577
  • 16
  • 155
  • 122
  • Oh! Thanks for the links, I didn't know CouchDB and Redis had this feature, I have mostly used MongoDB but I will probably switch to one of these then :) – Tug Dec 28 '11 at 20:41
  • Each db has its own purpose, Redis is good for fast, in memory data, pub-sub and queue features; CouchDB is great for replication, data-safety and _changes; MongoDB is really powerful when it comes to speed, making queries in a 'friendlier' way than regular map-reduce etc. – alessioalex Dec 28 '11 at 20:47
  • Sure :) but Still MongoDB could really benefit from this feature – Tug Dec 28 '11 at 21:17
1

Instead of triggers for case 4, you might want to hook into MongoDB replication.

Let's assume you have a replica set (you wouldn't run single mongod, would you?).

Every change is written to the oplog on the primary and then is replicated to secondaries.

You can efficiently pull new updates from the oplog, using tailable cursors. Note, this is still pull, not push.

Then your node.js will push these events to the clients.

Sergio Tulentsev
  • 226,338
  • 43
  • 373
  • 367
  • I believe tailable cursors only works when inserting new documents, is it not? What if I wanted to be aware of updates (like storing the messages as an array in a room document) ? – Tug Dec 28 '11 at 20:50
  • No, you can get updates too. That's how replication works. Every write ends up in oplog. – Sergio Tulentsev Dec 28 '11 at 20:51
  • And you don't have to use tailable cursors. You can use regular ones. But it will be less efficient. – Sergio Tulentsev Dec 28 '11 at 20:52
  • ok thx for the advice, I'll have a look at how oplog works, it should be interesting but I will probably switch to another DB proposing a solution out of the box :) – Tug Dec 28 '11 at 21:04