19

I have seen answers in couple of threads but didn't work out for me and since my problem occurs occasionally, asking this question if any one has any idea.

I am using jedis version 2.8.0, Spring Data redis version 1.7.5. and redis server version 2.8.4 for our caching application.

I have multiple cache that gets saved in redis and get request is done from redis. I am using spring data redis APIs to save and get data.

All save and get works fine, but getting below exception occasionally:

Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool | org.springframework.data.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the poolorg.springframework.data.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:198)
org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:345)
org.springframework.data.redis.core.RedisConnectionUtils.doGetConnection(RedisConnectionUtils.java:129)
org.springframework.data.redis.core.RedisConnectionUtils.getConnection(RedisConnectionUtils.java:92)
org.springframework.data.redis.core.RedisConnectionUtils.getConnection(RedisConnectionUtils.java:79)
org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:191)
org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:166)
org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:88)
org.springframework.data.redis.core.DefaultHashOperations.get(DefaultHashOperations.java:49)

My redis configuration class:

@Configuration
public class RedisConfiguration {

@Value("${redisCentralCachingURL}")
private String redisHost;

@Value("${redisCentralCachingPort}")
private int redisPort;

@Bean
public StringRedisSerializer stringRedisSerializer() {
  StringRedisSerializer stringRedisSerializer = new StringRedisSerializer();
  return stringRedisSerializer;
}

@Bean
JedisConnectionFactory jedisConnectionFactory() {
  JedisConnectionFactory factory = new JedisConnectionFactory();
  factory.setHostName(redisHost);
  factory.setPort(redisPort);
  factory.setUsePool(true);
  return factory;
}

@Bean
public RedisTemplate<String, Object> redisTemplate() {
  RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();
  redisTemplate.setConnectionFactory(jedisConnectionFactory());
  redisTemplate.setExposeConnection(true);
  // No serializer required all serialization done during impl
  redisTemplate.setKeySerializer(stringRedisSerializer());
  //`redisTemplate.setHashKeySerializer(stringRedisSerializer());
  redisTemplate.setHashValueSerializer(new GenericSnappyRedisSerializer());
  redisTemplate.afterPropertiesSet();
  return redisTemplate;
}

@Bean
public RedisCacheManager cacheManager() {
  RedisCacheManager redisCacheManager = new RedisCacheManager(redisTemplate());
  redisCacheManager.setTransactionAware(true);
  redisCacheManager.setLoadRemoteCachesOnStartup(true);
  redisCacheManager.setUsePrefix(true);
  return redisCacheManager;
 }

 }

Did anyone faced this issue or have any idea on this, why might this happen?

Bikas Katwal
  • 1,895
  • 1
  • 21
  • 42
  • Please post the full stack trace. – mp911de Apr 19 '17 at 10:13
  • @mp911de: Will provide once I have data. We have restarted our server so do not have logs, and since this issue occurs occasionally, guess I might need to wait. – Bikas Katwal Apr 20 '17 at 11:48
  • That's not the full stack trace. The reason for the connection failure is usually at the very end (a cause of `JedisConnectionException `). – mp911de Apr 26 '17 at 15:46
  • I cannot print the stack trace in my application. So I tried to capture it from Throwable.getStackTrace() and the logged it. Which gives me above stack trace. There is more in stacktrace, but that's my application methods which i didn't included. That's all I have. – Bikas Katwal Apr 28 '17 at 08:04

4 Answers4

6

We were facing the same problem with RxJava, the application was running fine but after some time, no connections could be aquired from the pool anymore. After days of debugging we finally figured out what caused the problem:

redisTemplate.setEnableTransactionSupport(true)

somehow caused spring-data-redis to not release connections. We needed transaction support for MULTI / EXEC but in the end changed the implementation to get rid of this problem.

Still we don't know if this is a bug or wrong usage on our side.

woezelmann
  • 1,355
  • 2
  • 19
  • 39
  • I had the same problem, but in my case I'm using RxJava. The transaction support classes of spring-data-redis seems to use ThreadLocal variables, which will not work in my case. Are you using any asynchronous frameworks? – Thomas Aug 18 '17 at 09:44
  • @Thomas it was also RxJava in our case, I updated my answer. – woezelmann Aug 22 '17 at 12:44
  • @Thomas Just saw your answer, been some time i posted this questions. I did solve this problem by playing with configurations, added below config, which solved my problem: poolConfig.setMaxIdle(30); poolConfig.setMinIdle(10); – Bikas Katwal Aug 24 '17 at 08:47
5

I moved from redis.template to plain jedis. Added below configuration(can be added in redis template too) for pool and don't see any exception now:

jedisPoolConfig.setMaxIdle(30);
jedisPoolConfig.setMinIdle(10);

for redis template:

jedisConnectionFactory.getPoolConfig().setMaxIdle(30);
jedisConnectionFactory.getPoolConfig().setMinIdle(10);

Same above config can be added in redis template too.

Community
  • 1
  • 1
Bikas Katwal
  • 1,895
  • 1
  • 21
  • 42
  • 2
    Well it seems that it worked for me also, but I am not sure why. Would you mind to elaborate? Basically my problem is what if my app will run out of idle connections. (current_connections > max_idle) – freefall Jul 27 '20 at 14:40
0

The problem is with the Redis configuration

For me, I was using this property for my local, when I commented on this property, the issue got resolved

#spring.redis.database=12

The correct property will be

spring.redis.sentinel.master=mymaster
spring.redis.password=
spring.redis.sentinel.nodes=localhost:5000

CodingBee
  • 1,011
  • 11
  • 8
0

I fixed mine by changing this in my application.yml file:

 redis:
   password: ${REDIS_SECRET_KEY: null}

to this:

password: ${REDIS_SECRET_KEY:}