The memory optimal setting of rails app (on Phusion passenger/nginx)

Question

I have a rails app using renv/Phusion Passenger/nginx on Ubuntu. My server has a small memory size, and the rails app is rarely invoked, so I need to find a memory optimal configuration on Phusion passenger/nginx.

Setup nginx.conf

I tried to minimize the memory usage, so I keep only 1 pool, and tried to minimize the idle time. I got information from this site: https://www.phusionpassenger.com/library/config/nginx/reference/#passenger_pool_idle_time

user www-data;
worker_processes 1;
pid /run/nginx.pid;

events {
    worker_connections 768;
}

http {
    ...
    ##
    # Phusion Passenger config
    ##
    # Uncomment it if you installed passenger or passenger-enterprise
    ##

    passenger_max_pool_size 1;
    passenger_pool_idle_time 1;
    passenger_root /home/ubuntu/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/passenger-5.0.21;
    passenger_ruby /home/ubuntu/.rbenv/shims/ruby;

    ##
    # Virtual Host Configs
    ##

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

Setup rails.conf

I have this configuration to rails.example.com is connected to a rails app.

server {
    listen 80;
    server_name rails.example.com;
    passenger_enabled on;
    passenger_app_env development;
    root /home/ubuntu/webapp/rails/passenger-ruby-rails-demo/public;
}

Checking the memory usage with htop, I have this map.

However, the information shown is not exactly the same as the one from sudo passenger-memory-status command.

The htop report has four Passenger RubuApps (with 3317, 3319, 3314, 3320 pids), but passenger-memory-status only shows 3314 pid. The same is true with other processes (core, watchdog, ust-router). I think htop shows the correct memory usage.

What makes the differences in memory usage information? I'm also curious about

How to minimize the memory usage of Phusion passenger with nginx?
Is there a way to kill passenger processes after a certain amount of time when there is no rails app invocation?
Other than Phusion passenger, what might be the way to use least amount of memory?

EDIT

Modifying the setup of htop to show in tree format and show threads in different color results in showing one process.

EDIT2

Passenger AppPreloader consumes the memory as much as RubyApp does.

After killing the AppPreloader, the memory consumption is smaller, and the app seems to be working fine.

I'm not sure if AppPreloader is absolutely necessary (for optimal memory usage) and is there an option not to use it.

score 2 · Accepted Answer · answered Nov 08 '15 at 12:33

Passenger author here. It is in fact passenger-memory-stats that shows the most correct measurement, not htop. See https://www.phusionpassenger.com/library/indepth/accurately_measuring_memory_usage.html for more information.

Furthermore, htop not only displays processes, but also threads. That is the reason why you see 4 entries in htop and 1 in passenger-memory-stats is because. Thus, passenger-memory-stats really is the most accurate.

As for your how to shutdown an app after a certain amount of time: Passenger already does that by default, but you mean want to tweak things a little. Have a look at these two config options:

Unicorn does not use less memory than Passenger. They are about the same because most memory is occupied by Rails and by the app, so it does not really matter. Passenger has a ton more features, tooling and documentation than Unicorn too.

The Passenger's AppPreloader is the one that makes me think Passenger uses more memory than Unicorn. I updated my post. — prosseek, Nov 08 '15 at 15:36
You can disable the AppPreloader. Set [passenger_spawn_method](https://www.phusionpassenger.com/library/config/nginx/reference/#passenger_spawn_method) to `direct`. — Hongli, Nov 08 '15 at 15:48
Also, read https://www.phusionpassenger.com/library/indepth/ruby/spawn_methods/ to learn what the AppPreloader is. — Hongli, Nov 08 '15 at 15:52

score 0 · Answer 2 · edited May 23 '17 at 12:15

I tried with unicorn server to launch rails app. The default WEBRrick server seems to be too slow and only for development purposes, so I skipped it.

I used rails new simple to create a skeleton. The setting files are as follows.

config/unicorn.rb

# set path to application
app_dir = File.expand_path("../..", __FILE__)
shared_dir = "#{app_dir}/shared"
working_directory app_dir

# Set unicorn options
worker_processes 1
preload_app true
timeout 30

# Set up socket location
#listen "#{shared_dir}/sockets/unicorn.sock", :backlog => 64
listen "/tmp/unicorn.sock", :backlog => 64

# Logging
stderr_path "/var/log/unicorn/stderr.log"
stdout_path "/var/log/unicorn/stdout.log"

# Set master PID location
pid "/tmp/unicorn.pid"

/etc/nginx/site-enabled/rails_unicorn.conf

upstream app {
    server unix:/tmp/unicorn.sock fail_timeout=0;
    keepalive 8;
}

server {
    listen 80;
    server_name rails.example.com;
    passenger_enabled on;
    passenger_app_env development;
    root /home/ubuntu/webapp/rails/simple/public;

    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;

        proxy_pass http://app/;
        proxy_redirect off;
    }
}

Gemfile

gem "unicorn-rails" added. For Rails 4.2.4, gem "rack-handlers"

Hint from How can I use unicorn as "rails s"?

/etc/init/unicorn_rails.conf

# simple uWSGI script
# http://uwsgi-docs.readthedocs.org/en/latest/Upstart.html

description "uwsgi tiny instance"
start on runlevel [2345]
stop on runlevel [06]

respawn

# https://stackoverflow.com/questions/14823972/upstart-node-js-working-directory
script
    chdir /home/ubuntu/webapp/rails/simple
    #exec /home/ubuntu/.rbenv/shims/rails server unicorn 
    exec /home/ubuntu/.rbenv/shims/unicorn -c config/unicorn.rb -D
end script

I have smaller memory footprint, but it's interesting to see that ruby still requires larger memory size, compared to the Python/uwsgi.