0

I am using delayed_job to perform as simple task: Open a cvs file(it contains a list of emails) and print every line to a debug log.

Since the file is big, that should take some time.

I enqueue the job with CSVJob.perform_later("test")

This is the active_job:

class CVSJob < ApplicationJob

  queue_as :default

  def perform(job_name)

    job_logger = Logger.new("#{Rails.root}/log/delayed_job.log")

    [..code omitted...]
    CSV.foreach(file_path, headers: true, col_sep: ",") do |row|
        sleep 0.05
        job_logger.debug "------------------------------->"
        job_logger.debug row.inspect
      end
    end

  end

Now, if I login into the rails console and type:

>Delayed::Job.all

I can see the job running.

But when I delete the job with Delayed::Job.delete_all and no job appears in the delayed_jobs table, the job is still running.

Or, I can find the job id of the job and i can do a simple job.destroy! and the job is deleted from the table.

But(!!) the job is still running! because I see the debug log still executing by doing a simple

    tail -f delayed_job.log

Job stops performing when it goes through the whole csv file.

The only way to really "kill" the job is to restart the delayed_job worker. But this defeats the whole purpose of controlling the execution of the job in every aspect.

Is there something that I am doing wrong?

Thank you in advance

Nick_K
  • 541
  • 6
  • 22
  • 2
    The delayed job table is just the queue for the jobs -- once the worker picks it up it will run until completion. Not sure what you are trying to do, but that's the whole point of background jobs -- running until they are done. – dbugger Jun 29 '23 at 13:22
  • Thank you @dbugger for your reply. Parsing the csv takes about 5 minutes and eventually I would like to update a progress bar with turbo_stream. First, I would like to be able to stop the execution once the job starts. That's what I want to do right now. – Nick_K Jun 29 '23 at 13:36
  • I am looking for something similar to this: https://stackoverflow.com/questions/25889699/sidekiq-stop-one-single-running-job – Nick_K Jun 29 '23 at 13:58
  • 1
    As you can see from that answer -- it's not a common practice for good reasons. A job is a discrete unit of work. You can have the job broadcast out it status to a SNS queue or database table or redis cache and have your UI poll from that -- but stopping and starting the job is not something you want to get into the habit of doing. – dbugger Jun 29 '23 at 14:49
  • Ok. I was thinking about the same thing too. Perhaps, let's create a semaphore and based on its value, a certain action should be happen in perform() method (like exiting the loop). Thank you. Your answer got me thinking because I didn't know if it was me or delayed_job's feature. Have a good day. – Nick_K Jun 29 '23 at 15:28

0 Answers0