1

I have a rails app which deployed on ec2 instance with nginx and passenger. For background job, i had used sidekiq with redis. I have 1 gb memory on server. I also have set max_pool_size to 1 for passenger in nginx config, and also set sidekiq concurrency to 1.

when passenger get multiple requests it aquire maximum memory and sidekiq was killed by the reason of out of memory.

Passenger and sidekiq take approx 54% of memory at ideal position and did not free memory after complete the process. below the log at ideal position:

ps aux --sort=-%mem | awk 'NR<=10{print $0}'
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
ubuntu   13975  1.5 24.8 774064 251908 ?       Sl   11:04   0:01 Passenger RubyApp: /var/www/dairylac/Quickmilk/current/public (staging)
ubuntu   14108  5.1 21.4 914332 217596 ?       Sl   11:04   0:03 sidekiq 4.1.2 Quickmilk [0 of 1 busy]
ubuntu   13858  3.8 19.9 491072 202544 ?       Sl   11:03   0:04 Passenger AppPreloader: /var/www/dairylac/Quickmilk/current
postgres 13986  0.2  4.4 192528 45048 ?        Ss   11:04   0:00 postgres: enbake quickmilk_dev_one 127.0.0.1(32902) idle
postgres  1489  0.0  3.5 185752 36044 ?        Ss   Aug29   0:06 postgres: writer process   
ubuntu   13952  0.0  1.9  42764 19376 ?        Ssl  11:04   0:00 /var/www/dairylac/Quickmilk/shared/bundle/ruby/2.1.0/gems/appsignal-1.2.5/ext/appsignal-agent
root     13811  0.1  1.6 677924 16744 ?        Sl   11:03   0:00 Passenger core
postgres 14117  0.0  1.4 187736 14316 ?        Ss   11:04   0:00 postgres: enbake quickmilk_dev_one 127.0.0.1(32924) idle
postgres 13959  0.0  1.4 187736 14308 ?        Ss   11:04   0:00 postgres: enbake quickmilk_dev_one 127.0.0.1(32900) idle

And when passenger and sidekiq get multiple requests then sidekiq killed by the reason of out of memory.

 ps aux --sort=-%mem | awk 'NR<=10{print $0}'
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
ubuntu   27463  3.9 33.8 1045284 343428 ?      Sl   13:11   0:05 sidekiq 4.1.2 Quickmilk [1 of 1 busy]
ubuntu   27466  1.5 31.3 841340 317708 ?       Sl   13:11   0:02 Passenger RubyApp: /var/www/dairylac/Quickmilk/current/public (staging)
ubuntu   27370  2.8 18.6 492812 189012 ?       Sl   13:11   0:04 Passenger AppPreloader: /var/www/dairylac/Quickmilk/current
postgres 27476  0.2  4.0 192540 41344 ?        Ss   13:11   0:00 postgres: enbake quickmilk_dev_one 127.0.0.1(33998) idle
postgres 27915 21.0  3.5 189828 36296 ?        Ss   13:13   0:03 postgres: enbake quickmilk_dev_one 127.0.0.1(34040) idle in transaction
postgres  1489  0.0  3.4 185752 34680 ?        Ss   Aug29   0:07 postgres: writer process   
ubuntu   27444  0.0  1.5  42764 15228 ?        Ssl  13:11   0:00 /var/www/dairylac/Quickmilk/shared/bundle/ruby/2.1.0/gems/appsignal-1.2.5/ext/appsignal-agent
root      1078  0.0  0.9 278696 10040 ?        S<sl Aug29   0:01 /usr/lib/snapd/snapd
redis     1069  0.0  0.8  39236  8976 ?        Ssl  Aug29   0:42 /usr/local/bin/redis-server 127.0.0.1:6379

I did not found why passenger and sidekiq was killed and how can i solve it.

Please give me any suggesstion.

Thanks in advance.

Bhupendra Singh
  • 11
  • 2
  • As per this answer: https://stackoverflow.com/a/726762/1010826, kernel may kill a process under _extreme resource starvation_. And this may as well be helpful: https://stackoverflow.com/a/726879/1010826 – Magnuss Aug 30 '17 at 09:48
  • But why sidekiq memory consumption increments according to times, whenever sidekiq worker is in ideal state. And how can we set maximum memory consumtion by sidekiq. – Bhupendra Singh Aug 30 '17 at 11:05
  • Your logs show `sidekiq 4.1.2 Quickmilk [1 of 1 busy]` that means a job is running - inspect that job and see why it's taking that memory. – Magnuss Aug 30 '17 at 11:11
  • Hi Magnuss, i shared two logs here, you can see in first logs sidekiq worker is free there. But still this take so much memory. – Bhupendra Singh Aug 31 '17 at 05:05
  • Seems that you have 1 gig or ram which is really not a lot. 200MiB of RAM for one RoR app is not unusual. How much did you expect it to be? – Magnuss Aug 31 '17 at 07:43
  • I have 1 gb RAM, But i did not understand why my server take below memory on ideal state: – Bhupendra Singh Aug 31 '17 at 12:24
  • Ruby doesn't reduce memory use nearly as much as native code when it's idle. It keeps all the loaded classes and constants in memory so that the next task to run doesn't have to load them from disk. As has been stated 200MB for an idle process isn't abnormal. If you want to reduce the absolute memory use you can set the [passenger_min_instances](https://www.phusionpassenger.com/library/config/nginx/reference/#passenger_min_instances) to 0, however it will then be slower to serve the first request after an idle period. – Camden Narzt Aug 31 '17 at 16:18

0 Answers0