What's the default timeout of ioredis send command for any redis call

Question

I am using ioredis with a node application, and due to some issues at cluster I started getting:

Too many Cluster redirections. Last error: Error: Connection is closed.

Due to which all of my redis calls failed and after a very long time ranging from 1sec to 130secs.

Is there any default timeout for ioredis library which it uses to assert the call after sending command to execute to redis server?

Higher failure time of range 100secs on sending commands to redis server, is it because the the high queue size at redis due to cluster failure?

Sample code :

this.getData = function(bucketName, userKey) {
  let cacheKey = cacheHelper.formCacheKey(userKey, bucketName);
  let serviceType = cacheHelper.getServiceType(bucketName, cacheConfig.service_config);
  let log_info = _.get(cacheConfig.service_config, 'logging_options.cache_info_level', true);
  let startTime = moment();
  let dataLength = null;
  return Promise.try(function(){
    validations([cacheKey], ['cache_key'], bucketName, serviceType, that.currentService);
    return cacheStore.get(serviceType, cacheKey);
  })
  .then(function(data) {
    dataLength = (data || '').length;
    return cacheHelper.uncompress(data);
  })
  .then(function(uncompressedData) {
    let endTime = moment();
    let responseTime = endTime.diff(startTime, 'miliseconds');
    if(!uncompressedData) {
      if(log_info) logger.consoleLog(bucketName, 'getData', 'miss', cacheKey, that.currentService,
        responseTime, dataLength);
    } else {
      if(log_info) logger.consoleLog(bucketName, 'getData', 'success', cacheKey, that.currentService,
        responseTime, dataLength);
    }
    return uncompressedData;
  })
  .catch(function(err) {
    let endTime = moment();
    let responseTime = endTime.diff(startTime, 'miliseconds');
    logger.error(bucketName, 'getData', err.message, userKey, that.currentService, responseTime);
    throw cacheResponse.error(err);
  });
};

Here logger.error(bucketName, 'getData', err.message, userKey, that.currentService, responseTime);

started giving response time of range 1061ms to 109939ms.

Please provide some inputs.

score 2 · Answer 1 · answered Mar 28 '19 at 08:53

2

As you can read from this ioredis issue, there isn't a per-command timeout configuration.

As suggested in the linked comment, you can use a Promise-based strategy as a workaround. Incidentally, this is the same strategy used by the ioredis-timeout plugin that wraps the original command in a Promise.race() method:

//code from the ioredis-timeout lib
return Promise.race([
      promiseDelay(ms, command, args),
      originCommand.apply(redis, args)
]);

So you can use the plugin or this nice race timeout technique to add a timeout functionality on top of the redis client. Keep in mind that the underlying command will not be interrupted.

answered Mar 28 '19 at 08:53

lifeisfoo

15,478
6
74
115

Thanks @lifeisfoo, yes that surely is a solution, was looking for any in built timeout option if any. Apart from that, can you help me why the above, code couldn't stop cascading effect of taking the whole application down ? I already had fallbacks in place for that. Is disabling enableOfflineQueue right strategy to avoid cascading effect ? And also along with it add timeout strategy using race or circuit breaker sort of solution ? – mohit3081989 Mar 28 '19 at 10:17
@mohit3081989 have you tried using the _enableOfflineQueue: false_ option? Does your commands now exit immediately when the connection is down? Have you checked all the available options for the cluster? https://github.com/luin/ioredis#cluster Documentation says: "_You should make sure retryDelayOnFailover * maxRedirections > cluster-node-timeout to insure that no command will fail during a failover_" – lifeisfoo Mar 28 '19 at 12:52
that's the plan, I am going to disable offline queue, and along with it I will put circuit breaker to make sure command exit at reasonable time. All the params are default by the way, so retryDelayOnFailover shouldn't be a issue . In default, offline queue is enabled so will make changes to disable it. Also I didn't find cluster-node-timeout param in entire library, so either that's miss documented or the param name is something else. – mohit3081989 Apr 03 '19 at 15:21

Stupid Man · Answer 2 · 2021-03-31T20:27:12.727

I was facing a similar issue which I have described in detail here: How to configure Node Redis client to throw errors immediately, when connection has failed? [READ DETAILS]

The fix was actually quite simple, just set enable_offline_queue to false. This was with Node Redis, so you'll have to figure out the equivalent for IORedis. Setting this to false, will make all commands throw an exception immediately which you can process in a catch block and continue instead of waiting for some timeout.

Do keep in mind that, with enable_offline_queue set to false, the commands that you issue while there's some connection issue with the server will never be executed.

What's the default timeout of ioredis send command for any redis call

2 Answers2

Linked

Related