1

We have a rails 4 app that has a scheduled rake task, which will spawn sidekiq workers, and we started to see errors:

An ActiveRecord::ConnectionNotEstablished occured in background
Exception
No connection pool for User

config/initializers/leaked_connections.rb:44:in `connection_with_forbid_implicit'
lib/ads_apis/ads_geo_targeter_base.rb:75:in `seeder_sync_targeting'

which is simply:

user = User.find(user_id)

we tried to lower the concurrency at sidekiq settings and increasing the pool size at the database settings, but hasn't fixed the issue, and we keep getting No connection pool for for models at the app.

Is there any hint to investigate this ?

simo
  • 23,342
  • 38
  • 121
  • 218
  • can you share your `leaked_connections.rb` file ? When have you noticed this to happen first ? When you say "increasing the pool size at database level" does it mean changing the db pool size of the rails app in `config/database.yml` or increasing the Database plan with more avaiable connections ? – Maxence Sep 04 '22 at 09:22
  • @Maxence please check `leaked_connections.rb at `https://gist.github.com/hopewise/61b1c7b9e752dc22901a04ef2ba65f15 the pool size is changed at the `config/database.yml` – simo Sep 04 '22 at 10:10
  • Your webserver is Puma ? – Maxence Sep 04 '22 at 14:01
  • I've just checked, its WEBrick – simo Sep 04 '22 at 14:26
  • Webrick is usually not a production server : https://stackoverflow.com/questions/10859671/webrick-as-production-server-vs-thin-or-unicorn . If your app is using Webrick in production then your app must be quite small with low traffic, and then the patch may really not be that helpful.. Also it writes on Rails core module ActiveRecord then I am not sure in what extend it prevents Sidekiq from running but I really advocate for using Puma and following the classic DB pool configuration – Maxence Sep 04 '22 at 14:33

1 Answers1

0

Seems like the patch is pretty old, we can see it mentionned in this article : https://medium.com/points-san-francisco/threading-with-rails-eeb843bfa20

The article mentions that the patch is not working for Rails 5 upwards. So if your Rails app is still stuck at Rails 4 it may be the reason.

Also the point of this patch is to alleviate the database load. But basically if your db pool is equal to Rails_max_thread (considering your webserver is Puma) then it may reach the max number of available database connections at some point and then require matching resources (RAM, computing etc ) for the database. Then I am not too sure in what aspect the patch would be that useful..

Default Rails max threads is 5, and this is often discussed https://twitter.com/nateberkopec/status/1564689914830229504?s=20&t=2uzgF8xYNRzgWtjEJuvfEQ but you may alleviate the database load by reducing the number of threads to something like 3 and increasing Puma concurrency (Puma workers)

Maxence
  • 2,029
  • 4
  • 18
  • 37
  • @Maxense how about forcing the same DB connection to be used for all active records calls within a sidekiq worker? ex: checkout a connection and checkin when we are done.. – simo Sep 05 '22 at 15:13
  • If Sidekiq concurrency is set to 10, then there can be 10 Sidekiq threads making database calls simultaneously. This is why the Rails db pool should at least match the Sidekiq concurrency. If you manage to get all Sidekiq threads to work with a limited number of DB clients (provided this is possible) Sidekiq performance would decrease. Although this is my knowledge, maybe you can raise a ticket on Sidekiq repo to get more accurate information. – Maxence Sep 05 '22 at 15:21
  • Thanks, for out setup, the sidekiq concurrency is even less than %25 of pool size for the database config pool, I will checkout.. – simo Sep 05 '22 at 15:24