Monitoring bundle exec unicorn_rails with bluepill

Question

Due to unicorn_rails complaining about different gem versions we moved to running bundle exec unicorn_rails... in our bluepill files. This change solved that particular problem and things start and stop but when we try sudo bluepill status we now get

unicorn(pix: XXXXXX): unmonitored

Which looks like bluepill is not monitoring the unicorn processes now. It will restart the child processes if I stop them but won't restart the parent process.

I've searched around but can't find much about this issue and was hoping someone could shed some light on it. The bluepill config file is

app_dir = "/opt/local/share/httpd/apps/xyz"
Bluepill.application('xyz', :log_file => "#{app_dir}/current/log/bluepill.log") do |app|
  app.process('unicorn') do |process|
    process.pid_file    = "#{app_dir}/shared/pids/unicorn.pid"
    process.working_dir = "#{app_dir}/current"

    process.stdout = process.stderr = "#{app_dir}/shared/log/unicorn.err.log"
    process.start_command = "bundle exec unicorn_rails -D -c #{app_dir}/current/config/environments/production/unicorn.rb -E production"
    process.stop_command = "kill -QUIT {{PID}}"
    process.restart_command = "kill -USR2 {{PID}}"

    process.start_grace_time = 8.seconds
    process.stop_grace_time = 5.seconds
    process.restart_grace_time = 13.seconds

    process.monitor_children do |child_process|
      child_process.stop_command = "kill -QUIT {{PID}}"

      child_process.checks :mem_usage, :every => 10.seconds, :below => 200.megabytes, :times => [3,5]
      child_process.checks :cpu_usage, :every => 10.seconds, :below => 50, :times => [3,5]
    end
  end

end

score 3 · Answer 1 · answered Aug 18 '11 at 15:42

When you run bundle exec, it sets up an environment and forks the unicorn_rails process. Bluepill ends up monitoring the original bundle exec process instead of unicorn which is why you see unmonitored.

I set up my bundler environment directly in bluepill and then execute unicorn_rails directly:

Bluepill.application('xyz') do |app|
  app.environment = `env -i BUNDLE_GEMFILE=#{app_dir}/Gemfile bundle exec env`.lines.inject({}) do |env_hash,l|
    kv = l.chomp.split('=',2)
    env_hash[kv[0] = kv[1]
    env_hash
  end

  app.process('unicorn') do |process|
    process.start_command = "unicorn_rails -D -c #{app_dir}/current/config/environments/production/unicorn.rb -E production"
  end
end

(Note: I omitted part of the above config file for clarity. Your config file looks good, just try adding the app.environment stuff above and removing bundle exec from your start command.)

This sets up the environment in bluepill by capturing the bundler environment variables using backticks, parsing the returned string into a hash and assigning it to app.environment.

We had implemented your solution for a while, but bluepill doesn't actually pay any attention to the pid of the process it launches when the process self-daemonizes, it just expects the pid file to be generated by the process it's running (i.e. unicorn). — d11wtq, Sep 16 '11 at 11:39

score 3 · Answer 2 · answered Sep 16 '11 at 11:35

I know this question is old, but I've been facing this problem for weeks. My suspicions were aroused when I realised that 'bluepill quit', followed by reloading the pill while unicorn was running allowed bluepill to consider the process 'up'.

@blt04's answer didn't help. Today I came to a realization. My grace start time of 8 seconds was not enough, because I had preload_app true in my unicorn config... and my app (Rails) takes 12 seconds to load, not 8.

Raising the start time to 30 seconds (wildly in excess of what was needed) solved the problem. Bluepill just says 'starting' for 30 seconds, then goes to 'up' correctly. Unicorn starts and runs as normal.

You'll want your restart time to be longer than it takes for Rails to start too.

Monitoring bundle exec unicorn_rails with bluepill

2 Answers2