My question is same this question: How to limit concurrent message consuming based on a criteria
However, I want to have X consumers to process messages at the time.
The scenario (I've simplified things):
- Many end users can start jobs (heavy jobs, like rendering a big PDF for example), from a front end web application (producer).
- The jobs are sent to a single durable RabbitMQ queue.
- Many worker applications (consumers) processes those jobs and write the results back in a datastore.
This fairly standard pattern is working fine.
The problem: if a user starts 10 jobs in the same minute, and only 10 worker applications are up at that time of day, this end user is effectively taking over all the compute time for himself.
The question: How can I make sure only two job per end user is processed at any time ?
Also, I do not want the front end application to block end users from starting concurrent jobs. I just want the end users to wait for their concurrent jobs to finish one at a time.
The solution?: Should I dynamically create one auto-delete exclusive queue per end users ? If yes, how can I tell the worker applications to start consuming this queue ? How to ensure two worker will consume from this queue ?
