0

We have end-user API to perform some long (30s-1h) CPU-bound operation. User can create task, poll task status many times and finally get result.

Internally we have queue for tasks and some workers who can handle those tasks.

Problem occurs when User1 create a lot of large tasks, and right after that User2 create one small task. User2 is expecting to get result quickly, but all workers are handling tasks from User1. Task from User1 is at the end of queue.

Expectation is handling tasks from both users in round-robin manner.

Is there some production-ready solution for solve such kind of problem?

(UPDATE) Found the same question: What are some queuing mechanisms for implementing round-robin queues?

Denis Sokolov
  • 85
  • 1
  • 8
  • 1
    I assume that it is possible to determine the size of the task before it is dispatched to the worker. Then you either need to have ability to pause and resume the long running task or have a separate pull of workers for small tasks. – Maxim Fateev Sep 24 '19 at 02:49

1 Answers1

1

Probably, one approach would be to introduce prioritization of tasks.

For example, depending on the estimated task execution time - distinguish small (S), medium (M) and large (L) ones. Then tasks with different priorities will be sent to different queues. So, finally, user can poll all priority queues (S, M and L) for his specific task and find the result quicker then by waiting in single queue.

There can be couple of variations of this approach. For example, user can distinguish S/M/L by himself and know which queue to poll exactly (this way eliminating polling all queues). Alternatively, there can be some dispatcher component/service, that will do it globally and notify users which queue they should poll.

Oleksii Zghurskyi
  • 3,967
  • 1
  • 16
  • 17