9

We are facing this specific issue using lettuce redis library. We are receiving too many RedisCommandTimeoutException. We have set a timeout of 2 secs in redis-cli and 10 ms in redis slow logs. While nothing gets logged in slowlogs our application keeps getting this timeout.

Code we are using is as follows

Duration timeout = 
Duration.ofMillis(applicationProperties.redisTimeOut);
RedisClient client = RedisClient.create(RedisURI.create(applicationProperties.redisUrl));
client.setDefaultTimeout(timeout);
RedisCommands<String, String> commands = client.connect().sync();

We have about 100 threads in our application wgich might be using this shared connection

Exception we receive is as follows

io.lettuce.core.RedisCommandTimeoutException: Command timed out
at io.lettuce.core.LettuceFutures.awaitOrCancel(LettuceFutures.java:114)
at io.lettuce.core.FutureSyncInvocationHandler.handleInvocation(FutureSyncInvocationHandler.java:62)
at io.lettuce.core.internal.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:80)
at com.sun.proxy.$Proxy11.hmget(Unknown Source)
Vicky Desai
  • 223
  • 1
  • 3
  • 9
  • 3
    Did you manage to figure this out? Having the same problem at the moment. – user2296600 Apr 19 '18 at 21:04
  • we have got a lead though not 100% sure if that is the only cause . Incase of network fluctuation it takes a good amount of time for library to recover and hence in most cases throws timeout exception. A good idea would be to use a lower value for timeout and implement retries – Vicky Desai Apr 20 '18 at 11:33
  • Yeah that's what I was trying to avoid having to do. I did manage to run some load testing of about 100 million requests to our Redis Cluster using lettuce with the connection pool. Have not had a single command timeout since. Have you considered trying that? – user2296600 Apr 20 '18 at 18:51
  • hey can you let me know about this in details as I believe that connection pooling will ensure that subsequent request don't die but how does it help to the request already initiated and impacted by network fluctuation. – Vicky Desai Apr 24 '18 at 11:13
  • I have such a problem. how do you think if the problem is the Redis itself? I mean there may be misconfigurations which makes the Redis not to be able to handle a large number of simultaneous requests. – Amin Heydari Alashti Oct 11 '18 at 03:58
  • any solution you found? I am facing the same problem on my localhost, so no reason to consider network legging. – roottraveller Aug 02 '19 at 06:39
  • Anyone figure this out? Happening to my prod environment and can’t figure out what’s causing the timeouts. – John M Apr 18 '21 at 23:52

1 Answers1

1

I had the same error and it was that redis fell suddenly and when doing the query it took a long time and several retries, I found this way, I think there is a better way, but it worked for me

@Bean
public LettuceConnectionFactory redisConnectionFactory() {
        RedisStandaloneConfiguration redisConf = new RedisStandaloneConfiguration();
        redisConf.setHostName(env.getProperty("spring.redis.host"));
        redisConf.setPort(Integer.parseInt(env.getProperty("spring.redis.port")));
        redisConf.setPassword(RedisPassword.of(env.getProperty("spring.redis.password")));

        LettuceConnectionFactory factory = new LettuceConnectionFactory(redisConf);
        factory.setTimeout(500L); //timeout to redis

        return factory;
    }
David Leonardo Bernal
  • 61
  • 1
  • 6
  • You basically increased the timeout, didn't your application became slow? And was redis went down in your case? I have these exceptions too but Redis wasn't stressed at all. So In my case I think I need to look at connection pooling. Still reading a bit on this. – Praan Aug 31 '23 at 05:24