delayed_job queue not being processed on Heroku

Question

I'm running a Rails 3 app with delayed_job. The issue I've come across is that though the app is correctly adding jobs to the queue, they are never being processed.

My Class

class User < ActiveRecord::Base
  after_create :send_welcome_email

  private

    def send_welcome_email
      UserMailer.delay.welcome_email(self)
    end
end

Inspecting things through the Rails console I can see that there are jobs in the queue. I can also see that there have been 0 attempts to perform the jobs. Spinning up a Heroku worker doesn't cause the jobs to be processed.

Any ideas?

Thanks!

Edit: Trying to clear the jobs queue as suggested below I ran rake jobs:clear and received the following error

rake aborted!
uninitialized constant Rake::DSL
/app/.bundle/gems/ruby/1.9.1/gems/rake-0.9.0/lib/rake/tasklib.rb:8:in `<class:TaskLib>'
/app/.bundle/gems/ruby/1.9.1/gems/rake-0.9.0/lib/rake/tasklib.rb:6:in `<module:Rake>'
/app/.bundle/gems/ruby/1.9.1/gems/rake-0.9.0/lib/rake/tasklib.rb:3:in `<top (required)>'
/app/.bundle/gems/ruby/1.9.1/gems/rake-0.9.0/lib/rake/rdoctask.rb:20:in `<top (required)>'
/app/.bundle/gems/ruby/1.9.1/gems/railties-3.0.7/lib/rails/tasks/documentation.rake:1:in `<top (requ
ired)>'
/app/.bundle/gems/ruby/1.9.1/gems/railties-3.0.7/lib/rails/tasks.rb:15:in `block in <top (required)>
'
/app/.bundle/gems/ruby/1.9.1/gems/railties-3.0.7/lib/rails/tasks.rb:6:in `each'
/app/.bundle/gems/ruby/1.9.1/gems/railties-3.0.7/lib/rails/tasks.rb:6:in `<top (required)>'
/app/.bundle/gems/ruby/1.9.1/gems/railties-3.0.7/lib/rails/application.rb:214:in `initialize_tasks'
/app/.bundle/gems/ruby/1.9.1/gems/railties-3.0.7/lib/rails/application.rb:139:in `load_tasks'
/app/.bundle/gems/ruby/1.9.1/gems/railties-3.0.7/lib/rails/application.rb:77:in `method_missing'
/app/Rakefile:7:in `<top (required)>'
/usr/ruby1.9.2/lib/ruby/1.9.1/rake.rb:2373:in `load'
/usr/ruby1.9.2/lib/ruby/1.9.1/rake.rb:2373:in `raw_load_rakefile'
/usr/ruby1.9.2/lib/ruby/1.9.1/rake.rb:2007:in `block in load_rakefile'
/usr/ruby1.9.2/lib/ruby/1.9.1/rake.rb:2058:in `standard_exception_handling'
/usr/ruby1.9.2/lib/ruby/1.9.1/rake.rb:2006:in `load_rakefile'
/usr/ruby1.9.2/lib/ruby/1.9.1/rake.rb:1991:in `run'
/usr/ruby1.9.2/bin/rake:31:in `<main>'

also try setting workers to 1 and then checking the `heroku logs` — David Tuite, May 23 '11 at 16:15

score 3 · Accepted Answer · answered May 24 '11 at 23:31

In the end, the problem turned out to be a bug in Rake 0.9.0. As Rails depends on Rake, running bundle install after this release of Rake broke my delayed jobs.

The fix is to add gem 'rake', '0.8.7' to your gemfile.

More details can be found below...

DHH's tweet: http://twitter.com/dhh/status/71966528744071169

Discussion in rails_admin's issue tracker: https://github.com/sferik/rails_admin/issues/428

score 2 · Answer 2 · answered May 23 '11 at 14:51

2

Two options for you:

These two gems watch delayed_jobs queue and automatically 'hires' background workers to process the thread:

Option 1: https://github.com/michelson/hirefire (This is what I am using because it supports mongoid)
Option 2: This branch of delayed_job developed by a heroku engineer has a similar autoscale feature: https://github.com/pedro/delayed_job/tree/autoscaling

Good luck

answered May 23 '11 at 14:51

Jonathan

16,077
12
67
106

Thanks, but I already have the hirefire gem as part of my project, yet the jobs aren't being processed. – Joshua Clanton May 23 '11 at 23:54
Did you set the HIREFIRE_EMAIL and HIREFIRE_PASSWORD heroku config settings? Do you have any exception monitoring going on? I wonder if an error is being thrown. Try this -- clear your queue. Then tail your heroku log, and add something to queue -- see if you get any useful output – Jonathan May 24 '11 at 13:52
I did set the email and password settings. No exception monitoring set up. Though I do get a strange error when trying to clear my queue. Adding that to my question. Thanks! – Joshua Clanton May 24 '11 at 22:35
Thanks for your help. Even though the initial answer wasn't what I was looking for, your questions set me on the right path. – Joshua Clanton May 24 '11 at 23:32
Glad I can help, Looks like you are new to SO -- I would accept this answer and open a new separate question on your rake issue. When posting your rake issue -- include the rake code. But it looks like you either need to have your rake task load the environment (task :mytask => :environment) or require a library referring to 'DSL' – Jonathan May 24 '11 at 23:42

score 0 · Answer 3 · answered May 23 '11 at 02:31

Are you running the delayed_job daemon in the same environment in which the application is running?

Try working the jobs manually using:
rake RAILS_ENV=development jobs:work

If you are running your app in production mode then you need to start delayed_job daemon as:
RAILS_ENV=production script/delayed_job start

score 0 · Answer 4 · answered May 24 '11 at 00:13

0

This is kinda a long shot but make sure you are using the same Ruby version on Heroku as you are on your local machine. I had a problem with delayed jobs on Heroku before because of this.

answered May 24 '11 at 00:13

David Tuite

22,258
25
106
176

delayed_job queue not being processed on Heroku

4 Answers4

Linked