Java scheduler which is completely independent of system time changes

Question

Was using Java Timer, Then switched to ScheduledExecutorService, but my problem is not fixed. As Tasks scheduled before system time change (through ntpd) are not executed on delay specified. Have no logs for same as nothing happens :(.

using jre 1.6.0_26 64 bit in my target on 64 bit linux.

Update: ScheduledExecutorService works fine on Windows. Problem is only on 64 bit Linux based system running 64 bit JVM. It works fine on 64 bit linux running 32 bit JVM...strange. Have not found any reference of same on any blogs either.

IBM's JAVA SDK has same problem (ibm-java-sdk-7.0-0.0-x86_64-archive.bin).

I had filed defect against JDK 7139684,It was accepted but has been closed and marked duplicate of 6900441. Please vote for it , if you feel its worth to get it fixed... I have no idea why its not been fixed since more than couple of years

Here is sample code I used to test this issue:

package test;

import java.io.IOException;
import java.util.Date;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;

/**
 * @author yogesh
 *
 */
public class TimerCheck  implements Runnable {

    ScheduledExecutorService worker;


    public TimerCheck(ScheduledExecutorService worker) {
        super();
        this.worker = worker;
        this.worker.schedule(this, 1, TimeUnit.SECONDS);
    }

    private static void update() {
        System.out.println("TimerCheck.update() "+new Date(System.currentTimeMillis()));
    }

    @Override
    public void run() {
            update();
            worker.schedule(this, 1, TimeUnit.SECONDS);
    }

    /**
     * @param args
     */
    public static void main(String[] args) {
        ScheduledExecutorService worker = Executors.newScheduledThreadPool(1);
        new TimerCheck(worker);
    }

}

Answer 1

There is bug in JVM for overall scheduling during Sytem time changes backward, which also impacts very basic Object.wait & Thread.sleep methods. It becomes too risky to keep Java App running when system time switches back by even certain seconds. You never know what your Java App will end up to.

So we have decided to:

• Write watch dog script (non Java :)) to check time change.
• if time switches back by certain amount, shutdown and restart Java app.

Another possibility was to move to 32bit JVM, But we are using JNI and then native libraries used on target platform are not 32bit compatible. Also from our experience 32bit JVM limits our target to 1.6G heap, which is not at all sufficient for us.

I know ours is not most elegant solution, but untill JVM is fixed or some better solution is found, there doesn't seem to be any other way.

Edited: We are also considering 1st suggestion from Chris in addition to above solution:

• Configure NTP to never have a big time jump. Only slew the time slowly. Only apply big time jumps manually during downtime.

Answer 2

I worked against the same problem and have not found a solution. I looked at Quartz and all of the built-in JRE methods. The nanotime methods may use per-CPU monotonic clocks, but you risk big jumps if the threads are migrated to another CPU, I believe. My conclusions were the following:

1. Configure NTP to never have a big time jump. Only slew the time slowly. Only apply big time jumps manually during downtime.
2. Use multiple shorter timeouts, and require two timeouts to declare a remote machine dead. This won't help if you have just one system timing itself, but can help with multiple systems
3. use a remote machine on purpose to send you periodic wakeups. If your own clock goes backward between wakeups, then you know all of your timers are going to fire late.

Basically, this is a huge pain but there are no silver bullets.