Example scenario: I have two APIs (Account API and API exposed to user) where Account API will provide me a list of accounts (approx min size 500 and may increase when more accounts are added). I want to fetch the accounts based on a filter and expose the data in the other API to the user.
I am using Spring @scheduled to schedule the job which fetches the accounts API every 30mins.
In a single Pod, the scheduler will execute the job with ease.
When the app is replicated to three Pods in Kubernetes, all the schedulers will wake at the same time and it's a duplication of the scheduler job.
Expected Behaviour:
I want the Schedulers to work in a synchronized fashion, where if an account is being processed by one scheduler, another scheduler should work on the next account. Basically, I want the scheduler to work like how a multi-threaded program would work.
Something like scheduler1 processes 200 accounts, scheduler 2 processes 200 accounts, and scheduler3 processes 100 accounts.
I am new to Spring boot application development and like to know if something like the above can be done.
I read about Shedlock, but it will enable only one schedular to run at a time. But I like to use all the schedulers and process the accounts faster
