How do I implement connection pooling for Redis-As-Rails-Cache (using Redis as rails cache) Rails 4.1?

Question

How can I add connection pooling to my Redis setup?

for some reason I can't seem to find any information on this. I'm using redis (3.2.0) set up like so: (you can assume resque_uri and cache_uri has been parsed ahead of this c)

for resque:

$REDIS_BACKGROUND_JOB =  Redis.new(:host => resque_uri.host, 
                                  :port => resque_uri.port, 
                                  :password => resque_uri.password)

later in the initialization sequence, we do

Resque.redis = $REDIS_BACKGROUND_JOB

We have a 2nd Redis instance with a different memory, connection limit, and eviction policy, for the Rails cache

$REDIS_CACHE = Redis.new(:host => cache_uri.host, 
                        :port => cache_uri.port, 
                        :password => cache_uri.password)

UPDATE 2017-02-16@20-17EST: I am running two redis instances, one for resque (1.25.2) and another for the caching layer using redis-rails (5.0.1). I have updated the setup examples above. I suspect something is misconfigured in the initialization or there is a fundamental setup mismatch.

UPDATE 2017-02-16@21-37EST:

environments/development.rb has

config.cache_store = :redis_store, 'redis://localhost:6379/1/cache_store'

Although I have found some examples that show me how to use ConnectionPool.new, I'm not sure how to hook it up to the Rails cache as configured in config/application.rb using this syntax: ```config.cache_store = :redis_store, "redis://localhost:6379/0/cache"``` — Jason FB, Feb 17 '17 at 01:59
Also wanted to add as a follow-up here that I am unable to get connection pooling working on Resque whatsoever; it errors out inside of the Resque code. (But I have been able to get a basic ConnectionPool working with ```gem 'connection_pool'``` when trying to instantiate a basic Redis.new object. However, as you will note in ```config/application.rb``` Rails' cache store is configured by ```config.cache_store``` and i'm not sure how to addd connection pooling to that) — Jason FB, Feb 17 '17 at 02:01
http://stackoverflow.com/questions/28113940/what-is-the-best-way-to-use-redis-in-a-multi-threaded-rails-environment-puma — Sooraj, Feb 17 '17 at 05:11
@Sooraj -- yes, I've seen that document but how does the Rails cache know to open that pooled connection instead of opening its own connections? This SO post refers to using Redis for Sidekiq, which is not my blocker. I do not understand how to pool the connection for the Rails cache (do I even need to?). I do understand this SO post shows how to poll the connection for Sideqik — Jason FB, Feb 18 '17 at 16:39
@Sooraj -- I will admit my question conflates two setups — one for Resque and another for the Rails cache. I believe the conflation of the two is my problem, but I haven't yet figured it out. The link you provided is an example for getting connection pooling working on Sideqik, so I'd like to disqualify it as an answer to this question as it does not answer the root of the problem: how do I configure the Rails cache to use a redis instance that is pooled? — Jason FB, Feb 18 '17 at 16:51
I believe redis-rails uses redis-rack which has connection pooling. You may follow the sidekiq method for connection pooling for resque. For Rails cache, you can just add the pool setting in the initializers: Rails.application.config.session_store :redis_store, { servers: { host: uri.host, port: uri.port || nil, password: uri.password, namespace: 'sessions', pool: 'test' }, pool_size: 5, pool: 'test' } — Sooraj, Feb 20 '17 at 05:45
Yes I think that works. I'm going to add an answer and answer my own question — Jason FB, Feb 27 '17 at 19:03

Jason FB · Accepted Answer · 2018-03-27T19:45:06.773

OK here's what I got working.

added to my Gemfile

gem 'connection_pool'

For the RAILS CACHE:

Initializing $REDIS_CACHE = Redis.new for the Rails cache is incorrect, as Rails does this inside of the config.cache_store setting in your environment files (config/environments/development.rb, config/environments/production.rb, etc)

And then my initialization in my environment files is as follows:

config.cache_store = :redis_store, ENV["REDISCLOUD_FOR_RAILS_CACHE_URL"], {pool_size: 10, pool_timeout: 10}

(you will note that ENV["REDISCLOUD_FOR_RAILS_CACHE_URL"] is set as the full URI, including protocol, username, password, host, and port. An alternative is to pass these each one as its own parameter here in a hash.)

How do I implement connection pooling for Redis-As-Rails-Cache (using Redis as rails cache) Rails 4.1?

1 Answers1