2

I have a task that I need to generate immediately after the request is created and get it done ASAP.

So for this purpose, I have created a /config/sidekiq.yml file where I defined this:

---
:queues:
  - default
  - [critical, 10]

And for the respective worker, I set this:

class GeneratePDFWorker
  include Sidekiq::Worker
  sidekiq_options queue: 'critical', retry: false

  def perform(order_id)
    ...

Then, when I call this worker:

GeneratePDFWorker.perform_async(@order.id)

So I am testing this. But - I found this post, where is said that if I want to execute the tasks immediately, I should call:

GeneratePDFWorker.new.perform(@order.id)

So my question is - should I use the combination of a (critical) queue + the new (GeneratePDFWorker.new.perform) method? Does it make sense?

Also, how can I verify that the tasks is execute as critical?

Thank you

Community
  • 1
  • 1
user984621
  • 46,344
  • 73
  • 224
  • 412
  • if you're just testing a worker .. i think running `GeneratePDFWorker.perform(@order.id)` on your rails console should do the work.. with your sidekiq daemon running on the machine you're testing – sa77 May 08 '16 at 14:15

3 Answers3

2

So my question is - should I use the combination of a (critical) queue + the new (GeneratePDFWorker.new.perform) method? Does it make sense?

Using GeneratePDFWorker.new.perform will run the code right there and then, like normal, inline code (in a blocking manner, not async). You can't define a queue, because it's not being queued.

Walking Wiki
  • 699
  • 4
  • 13
  • Correct, it won't run in the background, it will run in the same thread (i.e. ruby process) that called `GeneratePDFWorker.new.perform`, which really defeats the whole purpose of using Sidekiq in the first place. – Anthony E May 08 '16 at 14:42
1

As Walking Wiki mentioned, GeneratePDFWorker.new.perform(@order.id) will call the worker synchronously. So if you did this from a controller action, the request would block until the perform method completed.

I think your approach of using priority queues for critical tasks with Sidekiq is the way to go. As long as you have enough Sidekiq workers, and your queue isn't backlogged, the task should run almost immediately so the benefit of running your worker in-process is pretty much nil. So I'd say yes, it does make sense to queue in this case.

Also, you're probably aware of this, but sidekiq has a great monitoring UI: https://github.com/mperham/sidekiq/wiki/Monitoring. This should should make it easy to get reliable, detailed metrics on the performance of your workers.

Anthony E
  • 11,072
  • 2
  • 24
  • 44
0

should I use the combination of a (critical) queue?

Me:

Yes you can use critical queue if you feel so. A queue with a weight of 2 will be checked twice as often as a queue with a weight of 1.

Tips:

  • Keep the number of queues fewer as possible. Sidekiq is not designed to handler tremendous number of queues.
  • Also keep weights as simple as possible. If you want queues always processed in a specific order, just declare them in order without weights.

the new (GeneratePDFWorker.new.perform) method?

Me: No, using sidekiq in the same thread asynchronously is bad in the first place. This will hamper your application's performance as your application-server will be busy for longer. This will be very expensive for you. Then what will be the point of using sidekiq?

Shiva
  • 11,485
  • 2
  • 67
  • 84