How to bridge the testing using Resque with Rspec examples?

Question

I've a confusion while implementing Resque in parallel with Rspec examples. The following is a class with expensive method .generate(self) class SomeClass ... ChangeGenerator.generate(self) ... end

After implementing resque, the above class changed to the following and added a ChangeRecorderJob class.

class SomeClass
  ...
  Resque.enqueue(ChangeRecorderJob, self.id)
  ...
end

class ChangeRecorderJob
  @queue = :change_recorder_job

  def self.perform(noti_id)
    notification = Notification.find(noti_id)    
    ChangeGenerator.generate(notification)
  end
end

It works perfectly. But I have 2 concerns.

Before, my example spec used to test the whole stack of .generate(self) method. But now since I pushed that into Resque job, how can I bridge my examples to make that same test green without isolating? Or do I have to isolate the test??

And lastly, if I have 10 jobs to enque, do I have to create 10 separate job classes with self.perform method?

Answer 1

Testing asynchronous stuff like this is always tricky. What we do is:

• In our functional tests we make sure the job gets enqueued. Using mocha or something similar with an expectation is normally sufficient. If you want to run a test redis server, you could verify the correct queue grew and that the job params are correct. Although you're testing Resque itself a bit at that point.

• Jobs are tested in isolation as unit tests. Since they just have a class method called perform, your unit tests are quite simple. In your case you'd test that ChangeRecorderJob.perform does what you want. We tend to test that jobs are on the appropriate queue, that the params to the job are valid, and that the job does what we want.

• Now, to test everything together is the tricky part. I've done this two different ways and each has pros and cons:

  • Monkey-patch Resqueue.enqueue to run the job synchronously As of resque 1.14.0 you can use Resque.inline = true in your initializer instead of monkey-patching
  • Simulate a worker popping a job off the queue and actually run in a forked process

Running the job synchronously is by far the easier of the two. You'd just load something like the following in your spec_helper:

module Resque
  alias_method :enqueue_async, :enqueue

  def self.enqueue(klass, *args)
    klass.new(0, *args).perform
  end
end

As of resque 1.14.0 you can just set Resque.inline = true in your initializer instead of monkey-patching. If you're stuck on an older version of resque, the monkey-patch is necessary.

Note that because you are running synchronously here, you're going to incur the cost of your long-running job. Perhaps more importantly is you're going to be running in the same process so it's not a completely accurate representation of how your job is going to run.

To run the job in a forked worker, much like resque would, you'll need to do something like the following:

def run_resque_job(job_class, job_args, opts={})
  queue = opts[:queue] || "test_queue"

  Resque::Job.create(queue, job_class, *job_args)
  worker = Resque::Worker.new(queue)
  worker.very_verbose = true if opts[:verbose]

  if opts[:fork]
    # do a single job then shutdown
    def worker.done_working
      super
      shutdown
    end
    worker.work(0.01)
  else
    job = worker.reserve
    worker.perform(job)
  end
end

There's a slight delay in getting the worker to pop the job off the queue. And naturally you'll need to have a test redis server running so that the worker has a queue to pop off of.

I'm sure other people have come up with clever ways of testing resque jobs. These are what have been working for me.

Answer 2

Use resque_spec for unit testing.

describe "#recalculate" do
  before do
    ResqueSpec.reset!
  end

  it "adds person.calculate to the Person queue" do
    person.recalculate
    Person.should have_queued(person.id, :calculate).in(:people)
  end
end

And for your integration tests:

describe "#score!" do
  before do
    ResqueSpec.reset!
  end

  it "increases the score" do
    with_resque do
      game.score!
    end
    game.score.should == 10
  end
end

Answer 3

You will have to do two different tests. One for enquing, to make sure that the jobs are being enqueued into the Resque queues and the second for making sure that the jobs in the queue when picked up by the workers are being performed to your requirements.

No, you dont need to write 10 different perform methods. When you run Resque workers, they pick up the jobs from the queue and blindly call .perform method on your job. So, your job is expected to have the perform method.