I have this case where I have a Bunch of TimerTask objects that are scheduled as below. A daemon thread is created for each task. We started noticing that on one machine, that the task is not running on time, and after we dug a bit deeper, we found out the run() method is being called with a delay.
The case with one task is to run every four hours, so the first day it ran as scheduled (8:00:00, 12:00:00, 4:00:00, ... and so on), and all of a sudden later the task ran at (8:00:00, 12:00:00, 16:01:27, 20:01:27,...), then later in another day (8:01:27, 12:01:27, 16:03:33, 20:03:33,...), so what got the JVM to call the run() method with this time shift?
I took a heap dump and a bunch of thread dumps, the period of the task was 14400000 MS, which means that the task was not rescheduled or anything, and the timer's thread state was "java.lang.Thread.State: TIMED_WAITING" at 12:02:00, and the run method was called at 12:03:33, but it should have been called at 12:00:00.
Note1: The task execution time never exceeded 20 Minutes.
Note2: The shift affected all the running tasks, so a task that was supposed to run at 22:30:00 ran at 22:31:27 the day when the first shift happened.
I got the feeling that this could be an issue with Timers and TimerTasks, but I could not find a similar issue mentioned anywhere.
Thanks.
Timer timer = new Timer(true);
//task here is he encapsulated TimerTask object.
timer.schedule(task, task.getStartDateTime(), task.getPeriod());