node.js scalability with event timers (setTimeout)

Question

I'm building a turn-based text game with node.js and socket.io. Each turn has a timeout and after that the player loses the turn and it passes to the next player. I'm using the setTimeout function as I said in another question.

The problem is that I don't know how to scale that over multiple instances and maybe multiple servers. AIUI, if I set a timeout, I could only clear it in the same instance. So if a player loses his turn, for example, the timeout will be renewed with the other player turn, but this new player won't have access to the timer object to clear it, because it's running on the first player's instance.

I looked at Redis pub/sub feature (which I'll have to use anyway), but I didn't find anything about timed events or delayed publishing.

TL;DR, how can I keep an instance/server independent timer?

score 1 · Accepted Answer · answered Jun 03 '14 at 19:49

The solution I've found to this is to use some message system (Redis pub/sub in my case) to keep each player instance aware of the current status.

Each player has a worker instance which handles his own turn (this includes the timer). When it finishes, either via a move by the player or via the timeout, it advances the turn counter and informs all instances via the pub/sub with the new turn number. All of the instances receive the message and compares the turn number with its own player number. If it matches, then the instance handles the turn and the cycle repeats.

I'll try to provide an example (more of a pseudocode):

// pub & sub are Redis publisher & subscriber clients, respectively

function Game (totalPlayers, playerNumber) {
  this.turn = 0
  this.totalPlayers = totalPlayers
  this.playerNumber = playerNumber

  // Subscribe to Redis events
  sub.on('message', function (channel, message) {
    message = JSON.parse(message)

    switch(message.type) {
      case 'turn':
        this.onTurn(message.turn)
    }
  })

  sub.subscribe(this.channel, function() {
    this.checkStart()
  })
}

Game.prototype.checkStart = function () {
    // This checks if this instance  is for
    // the last player and, if so, starts the
    // main loop:
    if(this.playerNumber == this.totalPlayers - 1) {
      pub.publish(this.channel, JSON.stringify({type: 'turn', turn: 0})
    }
}

Game.prototype.onTurn = function(turn) {
  this.turn = turn
  if(this.turn == this.playerNumber) {
    this.timer = setTimeout(this.endTurn.bind(this), this.turnTime)
  }
}

Game.prototype.endTurn = function() {
  this.turn = (this.turn + 1) % this.totalPlayers
  pub.publish(this.channel, JSON.stringify({type: 'turn', turn: this.turn})
}

I had some problems with this approach and the main problem was the initial status, which wasn't quite right if the players connected almost at the same time. It's also a good idea to send information and make sure all instances are in sync.

I hope that I made this clear if someone is running into the same problem.

score 0 · Answer 2 · answered Sep 07 '20 at 10:47

A reliable independent timer could be done with Redis and its TTL option (plus its Pub/Sub mechanic).

//enable keyspace events:
redisClient.send_command('config', ['set', 'notify-keyspace-events', 'Ex']);

// add a key:
const key = '<some meaningful key string>';
redisClient.set(key, '<some note for the key, not usable though>');

// set the key to expire:
redisClient.expire(key, 100); // duration in seconds

// somewhere else in the code, subscribe to the 'expired' event:
const expiredSubKey = `__keyevent@${config.redis.db}__:expired`; // you need redis DB number here
redisClient.subscribe(expiredSubKey, () => {
    redisClient.on('message', async (channel, key) => {
        // channel is actually expiredSubKey, ignore it
        // 'key' is the key you've set up previously
    });
});

(more info: How to receive Redis expire events with node?)

Apart from being service-indepedent this technique has one more upside:

there is no polling involved, you don't need to check for the expired key regularly

it also has some downsides:

its a bit 'hacky', meaning it wasn't designed exactly for the purpose
I couldn't find a way to get the value on expiration event, so only a key can be used which is limiting
if you have multiple instances of a service (i. e. scaling) you'll have that many subscribers, thus the event will fire for every one of them. Sometimes this is not a problem, sometimes it is. This can actually be resolved through advanced Redis pub/sub.

Also you could use some third-party service for this. I was able to find a couple of them with free plans and reasonable API (though I'm using my own, described above ).

node.js scalability with event timers (setTimeout)

2 Answers2