4

I would like to build a job scheduler. So this job scheduler allows the user to configure:

  1. Job start time. (datetime value)
  2. Job frequency: minutes, hours, days, months, years (any integer value)

I have 2 options on how to build this scheduler:

  1. Use the C# Timer class. But this also means that I have to create a new Timer object for every scheduled job. I am planning to expose an API endpoint for the user to call and POST the starttime and frequency info. So when the endpoint is called, I will need to create a new Timer object. Will this even scale? How do I manage the Timer objects? I need to allow user to create, update and delete their jobs.

  2. Use the Azure Scheduler. However, I have a very large user database. I had a look at the pricing, the maximum total jobs that can run on 1 instance is 5 million jobs only. Plus, it is difficult for me to manage the running instances if I have more than 1 instance running. How do I decide to load balance multiple instances of schedulers?

NyamNyam
  • 320
  • 1
  • 3
  • 13
  • 1
    You can, with some math, handle multiple scheduled jobs with a single timer... – Zohar Peled Jan 23 '17 at 07:16
  • Yes, but it's not valid for my case because users can enter any integer for the job frequency. I will still end up creating different timers for different frequencies. I have a large user database, different users will want to configure their jobs differently. – NyamNyam Jan 23 '17 at 07:23
  • If you want to create it yourself, just run one timer and verify how much time left for the next iteration of you scheduled job and update it every time. E.g if you run your timer each second (may be too often) and hit a job that needs to start now, add one minute and store it as next execution time and on each timer tick you will check if this time has passed. Otherwise there are good libraries to do this, please see my answer below. – Ilya Chernomordik Jan 23 '17 at 08:13

4 Answers4

3

There are existing libraries that you can use, e.g. Quartz or Hangfire. The first one is rather simple to use library that I have used successfully, the latter has a UI in addition of running tasks, etc. that you can serve.

I am sure there are plenty of other libraries if those are not good enough.

Ilya Chernomordik
  • 27,817
  • 27
  • 121
  • 207
0

You could use Azure Worker roles, one worker role per job type, and different schedules for each job, users can create their own separate schedule for each job, picking from a list of predefined "job types".

Andrei Filimon
  • 1,138
  • 8
  • 12
  • can't afford to have that approach since the users can configure different recurring values. it will be too expensive – NyamNyam Jan 24 '17 at 09:35
0

For this you can use RabbitMQ. Basically, every scheduler is just producer-consumer concept over standard messenger. I recommend you to use some messenger for this task, because they can guarantee that your task executed if it was scheduled, even if your system was shutted down. It is self-balanced, stable and pretty much everything you need already implemented here.

More here: https://www.rabbitmq.com/tutorials/tutorial-one-dotnet.html

For your task you simply specify producer side which if running will enqueue tasks into RabbitMQ with some schedule without care about how they execute in messenger. This way you can add, delete, edit, read schedules in your producer. Also, a little advice, instead of frequency terms use cron-expressions. It is widely support, easy to understand concept for specifying human-readable schedules.

More here: https://crontab.guru/

eocron
  • 6,885
  • 1
  • 21
  • 50
0

Derek here from Azure Scheduler.We have enterprise customers using Scheduler similar to the scenarios you described, depending on what your user/job profile looks like, it should be easy to come up with a good way to partition them into different job collections.

How many jobs do you expect tot have? Sounds like you need much more than the 5 million we support in a single job collection with P20 plan. I'd like to better understand your scenario and help you decided whether Azure Scheduler is the right solution.

You can reach me at Derek.Li (at) microsoft dot com.

Derek Li
  • 3,089
  • 2
  • 25
  • 38