Extend ActiveJob with DelayedJob queue in Rails 5.x

Question

It doesn't seem possible to access the delayed_job object from within an activeJob.

I've work around that so far, by adding a context as an extra argument (and removing it on call), so I can add any extra information I need (e.g. multi tenancy). This get coded in the handler column as any other arguments.

I want to find duplicates though, and there are some issues when using this approach. So I'd like to define an extra job id j_id column, which will be used to find duplicates.

As I can't access the delayed_job, I can't find a way to populate it.

This is what I did so far:

# already added column j_id to delayed_job table in migration file
class ApplicationJob < ActiveJob::Base
 before_enqueue :add_context

  def add_context
    # How can I set the j_id from here?
  end

Answer 1

Rails's ActiveJob is similar to ActiveRecord, in that they both provide a stable API to a backend (a DB in AR's case and a queueing backengd for AJ). DelayedJob is just one of AJ's backends just like MySQL or Postgresql are backends for AR.

If you dig into DelayedJob's code (and you are using ActiveRecord as a backend to DelayedJob through the delayed_job_active_record gem) you will find that Delayed::Job is just another ActiveRecord::Base subclass, just like any of your models.

What that means is that you have access to the all DelayedJob objects through the Delayed::Job class, with all the find and validation methods you can expect from ActiveRecord.

#<Delayed::Backend::ActiveRecord::Job:0x00007f00889c3210
 id: 1,
 priority: 0,
 attempts: 0,
 handler:
  "--- !ruby/object:ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper\n" +
  "job_data:\n" +
  "  job_class: ApplicationJob\n" +
  "  job_id: 5f7539a3-120a-45b4-a6d7-12ec8c6cad3d\n" +
  "  provider_job_id: \n" +
  "  queue_name: default\n" +
  "  priority: \n" +
  "  arguments: []\n" +
  "  executions: 0\n" +
  "  exception_executions: {}\n" +
  "  locale: en\n" +
  "  timezone: UTC\n" +
  "  enqueued_at: '2020-03-02T05:38:22Z'\n",
 last_error: nil,
 run_at: Mon, 02 Mar 2020 05:38:22 UTC +00:00,
 locked_at: nil,
 failed_at: nil,
 locked_by: nil,
 queue: "default",
 created_at: Mon, 02 Mar 2020 05:38:22 UTC +00:00,
 updated_at: Mon, 02 Mar 2020 05:38:22 UTC +00:00>

This class is backed by a table called delayed_jobs, as you could expect from any AR class, so you can generate migrations that add or change columns on that table (add j_id):

rails g migration AddJIdToDelayedJobs j:string

To add a validation you could create config/initializers/delayed_job.rb and monkey patch the class.

module Delayed
  class Job
     validates :j_id, uniqueness: true                                                                                                                                  
  end                                                                                                                                       
end

The only ting left is to add data to the j_id column when the job is enqueued. This way you could add whatever information makes your jobs unique, it could be some other model's ID or the jobs parameters or anything you need. DelayedJob provides a cool plugin solution so we can use that to plug in to a jobs life-cycle and add the information to j_id when the job gets enqueued. Back in the config/initializers/delayed_job.rb lets add the new plugin class:

module Delayed
  class UniqueJobsPlugin < Plugin
    callbacks do |lifecycle|
      lifecycle.before(:enqueue) do |job|
        job.j_id = job #.anything inside the job, remember .handler is in YAML
      end
    end
  end
end
Delayed::Worker.plugins << Delayed::UniqueJobsPlugin

To get any info inside the YAML handler attribute you can do handler_attributes = YAML.load(job.handler)

Now if the j_id isn't unique enqueue will fail with the usual AR uniqueness error which you can handle in the controller that enqueued the job, or wherever you are enqueueing the job from.