3

I am using Redis in an Express application. My app is both a publisher and consumer of streams, using a single redis connection (redis.createClient). I have a question on the best way to manage a perpetual subscription (with xreadgroup). Currently I am doing this:

    const readStream = () => xreadgroup('GROUP' appId, consumerId, 'BLOCK', 1, 'COUNT', 1, 'STREAMS' key, '>')
        .then(handleData)
        .then(() => setImmeadiate(readStream));

Where xreadgroup is just a promisified version of node-redis' xreadgroup.

My question - What is the appropriate usage of BLOCK? If I block indefinitely or for a long period, then my client cannot publish any messages (with xadd) until it is unblocked, or the block times out. Since I must use some sort of loop/recursion in order to keep reading events, BLOCK appears to be fairly unnecessary; can I just leave it off and is this the expected usage?

Likewise, is using setImmeadiate appropriate or would process.nextTick or an async loop be preferred?

There is very little documentation in node-redis and the few examples simply read the messages once after blocking and do not produce/consume on the same client.

Alexey Zimarev
  • 17,944
  • 2
  • 55
  • 83
jramm
  • 6,415
  • 4
  • 34
  • 73

1 Answers1

0

Not an expert on the subject, but I'd like to share some thoughts that might help.

  1. I'm not sure if node-redis can "stack" multiple commands, meaning - will it be able to fire new commands while waiting for the XREADGROUP to complete?

  2. From your description, seems like that's what's happening. In that case, I suggest you create a dedicated connection to call XREADGROUP - this way you can publish and listen without blocking one another.

  3. You don't need to use the BLOCK; but if your goal is to listen to all events and wait for those not yet published, using it might be wise and will give you better performance while making less calls to redis.

  4. setImmediate is probably good, especially using BLOCK. If you don't use it, then it might be good to add a little bit of timeout between calls - without the BLOCK calls will answer return almost immediately. You can check this for more details.

Friendly reminder: don't forget to ACK your messages or use NOACK (might be ok depending on your use case):

consumer groups require explicit acknowledgment of the messages successfully processed by the consumer, via the XACK command.

The NOACK subcommand can be used to avoid adding the message to the PEL in cases where reliability is not a requirement and the occasional message loss is acceptable.

Source: https://redis.io/commands/xreadgroup

Rafael Paulino
  • 570
  • 2
  • 9