Sidekiq Batch - Scheduling jobs with perform_in and waiting for completion with callbacks

Question

I am considering using Sidekiq::Batch and learning from the following reference: https://github.com/sidekiq/sidekiq/wiki/Batches

I would like to know if the following is possible and what would happen in that case:

In the example below, I schedule a job using perform_in inside a batch.
The last of MyJob should be executed 5 hours later.

batch = Sidekiq::Batch.new
batch.on(:success, MyCallback)
batch.jobs do
    5.times do |index|
        MyJob.perform_in((1 * index).hour, args)
    end
end

By setting batch.on(:success, MyCallback), I can do something after all jobs have completed. Here, All jobs should be completed in 5 hours and a few minutes.

My questions are:

In this case, will the callback fire correctly after all jobs are completed in 5 hours and a few minutes (all job completed)?
In that case, where is the batch waiting and how can the batch know the all job has been completed?
Are there any negative effects if there are many batches waiting for a long time like this?

I appreciate any help as I am not familiar with Sidekiq, thank you in advance.

score 1 · Accepted Answer · answered May 10 '23 at 12:21

In this case, will the callback fire correctly after all jobs are completed in 5 hours and a few minutes (all job completed)?

Yes.

In that case, where is the batch waiting and how can the batch know the all job has been completed?

There is batch counter inside Redis that says 5 jobs. As each job finishes, that counter is decremented. On 0, the callback fires.

Are there any negative effects if there are many batches waiting for a long time like this?

Not at all. The batch is just data, it does not execute and consumes no resources while in progress, only the jobs and callbacks actually execute.

Sidekiq Batch - Scheduling jobs with perform_in and waiting for completion with callbacks

1 Answers1