8

After deploying our Rails app (4.0.9, Ruby 2.1.2), we notice requests to our app get hang after a while , usually 1 day or so.

Using the gem rack_timer, we're able to find out requests got stuck at ActiveRecord::QueryCache middleware.

Rack Timer (incoming) -- ActiveRecord::QueryCache: 925626.7731189728 ms

After removing it, our app seems to be back to normal. However, I understand the purpose of this middleware is to increase the performance, so removing is just a temporary solution. We're using mysql (5.1.67) with adapter mysql2 (0.3.13)

Update: Right after I posted this question, the server started hang again, this time requests were stuck at ActionDispatch::Routing::RouteSet

I, [2014-10-13T23:17:03.661346 #32498]  INFO -- : Rack Timer (Application Action) -- ActionDispatch::Routing::RouteSet: 3667661.2360477448 ms
I, [2014-10-13T23:17:03.661946 #32498]  INFO -- : Rack Timer (Application Action) -- ActionDispatch::Routing::RouteSet: 4373914.719343185 ms

Do you know any reason could cause this?

Thank you in advance.

VinhBS
  • 677
  • 8
  • 17
  • Check your other question, I have responded there. – Bowersbros Oct 21 '14 at 10:28
  • 1
    Did you find a solution? I am experiencing the same problem with postgres on heroku. – Pierre Olivier Martel Nov 10 '14 at 19:32
  • 1
    Any update on this? According to the feed there is "another question" but I cant find it.. – jalagrange Jan 09 '15 at 10:14
  • We're having issues like this too. It happens randomly. Did you find a solution? What causing it? It is the DB? Or is memcached? – Guillermo Mar 08 '15 at 14:35
  • I've been having this problem as well on heroku, please help! – Jared Mar 10 '15 at 12:13
  • I'm seeing this periodically as well on Heroku using rails 4.1.9 ruby 2.1.5 with postgres. – Dan Mar 11 '15 at 10:25
  • 1
    I found that if I increase the connection pool for the db these issues are gone. For example, I changed my default connection pool of 5 to 75 and I haven't had any issue since then. Of course, this means that more connections will open to the database, but it shouldn't be any issue if your db can handle the extra connections. – Guillermo Mar 16 '15 at 22:16
  • SImilar problem webrick / ruby 2.2.1 / postgres.... Increasing the pool didn't work for me. Any solutions available? – radubogdan Aug 28 '15 at 10:43

1 Answers1

2

The most possible cause is your connection with the database died (firewall, server configuration...) and Rails didn't notice, so a timeout happens inside ActiveRecord::QueryCache. Once Rails notices the connection died, it reconnects and the execution flow continues.

Try setting the param read_timeout in your database.yml file to something like 10 seconds and check your connection and server settings.

fjyaniez
  • 739
  • 6
  • 15
  • If I'm not wrong, when `QueryCache` is used, the database is not reached (that's why it is a cache), so I think it has to be related with the cache implementation being used (`mem_cache`, etc...) – Waiting for Dev... Sep 22 '16 at 07:32
  • Even if not used, a connection is established if I remember it correctly and the "hanging" issue can happen in that phase. – fjyaniez Sep 22 '16 at 11:48
  • read_timeout only available when using activerecord-jdbcmysql-adapter gem, btw. – Joe Sep 02 '21 at 16:42