16

The bug that annoys me is the same than this ticket. Basically, if you change the OS clock to a date in the past, all the thread that were sleeping at the time of the change won't wake up.

The application I am developping is meant to be running 24/24, and we would like to be able to change the OS date without stopping it (for example, to switch from summer time to winter time). What happens for the moment is that when we change the date to the past, then some parts of the application just freeze. I observed that on multiple machine, on Windows XP and Linux 2.6.37, and with a recent JVM (1.6.0.22).

I tried many Java sleeping primitives, but they all have the same behavior :

  • Thread.sleep(long)
  • Thread.sleep(long, int)
  • Object.wait(long)
  • Object.wait(long, int)
  • Thread.join(long)
  • Thread.join(long, int)
  • LockSupport.parkNanos(long)
  • java.util.Timer
  • javax.swing.Timer

Now, I am out of idea to work around this problem. I think there is nothing I can do to prevent the sleeping threads to freeze. But I would like, at least, to warn the user when a dangerous system clock change is detected.

I came up with a monitoring thread that detects such changes :

    Thread t = new Thread(new Runnable() {
        @Override
        public void run() {
            long ms1 = System.currentTimeMillis();
            long ms2;
            while(true) {
                ms2 = ms1;
                ms1 = System.currentTimeMillis();
                if (ms1 < ms2) {
                    warnUserOfPotentialFreeze();
                }
                Thread.yield();
            }                    
        }
    });
    t.setName("clock monitor");
    t.setPriority(Thread.MIN_PRIORITY);
    t.setDaemon(true);
    t.start();

The problem is that this makes the application grow from 2% CPU usage to 15% when idle.

Do you have an idea to work around the original problem, or can you think of another way to monitor the appearance of thread freeze ?

Edit

Ingo suggested not to touch the system clock. I agree that it's generally not needed. The problem is that we don't control what our clients do with their computers (we plan to sell hundred of copies).

Worse : one of our machine exhibits this problem without any manual intervention. I guess the OS (Windows XP) regularly synchronizes its clock to the RTC clock, and this makes the OS clock go back in time naturally.

Epilogue

I found out that some statements in my question were wrong. There are actually two separate causes involved in my initial problem. Now, I can say two things for sure :

  1. On my machine only (archlinux with kernel 2.6.37 with an OpenJDK 64 bits 1.6.0_22), Thread.sleep, Object.wait, Thread.join, LockSupport.parkNanos have the same problem : they wake up only when the system clock reaches the "target" time of awakening. However, a simple sleep in my shell does not exhibit the problem.

  2. On all the machines I tested (included mine), java.util.Timer and java.swing.Timer have the same problem (they are blocked until the "target" time is reached).

So, what I've done is that I replaced all the java's Timer by a simpler implementation. This solves the problem for all the machines but mine (I just hope my machine is an exception more than a rule).

Kara
  • 6,115
  • 16
  • 50
  • 57
barjak
  • 10,842
  • 3
  • 33
  • 47
  • 1
    Yes: do not change the OS clock. This is absolutely not needed. – Ingo May 06 '11 at 09:47
  • 24/24? Also are you sure that this bug happens during daylight savings time changes? What about NTP to keep the clocks sync'd; does that affect it? – trojanfoe May 06 '11 at 09:49
  • I haven't done much tests. It seems that NTP does not make the time go back, but rather slows down the clock until is reaches the right date. – barjak May 06 '11 at 10:04
  • 1
    Here's a crazy idea...what if you create some external app in another language (like C or whatever else you prefer) that does nothing but wait for a specified duration and then exit. Then instead of calling `Thread.sleep()` in Java, you can spawn a new instance of this external process, and then call `waitFor()` on it. This will "sleep" the Java thread for all practical purposes, and so long as your external app is able to sleep for the correct duration, it will resume at the correct time without getting frozen and without thrashing the CPU. – aroth May 06 '11 at 10:05
  • @aroth : not so crazy... It may be the only way to sleep in a safe way. Please consider posting your idea in an answer instead of a comment. – barjak May 06 '11 at 10:19
  • @barjak - Okay, I appended the suggestion to my original answer below. – aroth May 06 '11 at 10:29
  • That bug is marked as 'Fix Delivered' (in 1.6) - are you sure it's the same bug? – artbristol May 06 '11 at 10:29
  • @artbristol : there are differences between the ticket and what I observe : 1) I have this problem on Windows 2) I can see two different behaviors on two machines, one where the threads awake when the time is then put in the future, one where they don't. – barjak May 06 '11 at 11:07

5 Answers5

9

According to the bug ticket, your threads aren't frozen, they will resume once the clock catches up to where it was before it was modified (so if they moved it back an hour, your threads will resume in 1 hour).

Of course, that is still not very useful. The root cause seems to be that Thread.sleep() resolves to a system call that puts the thread to sleep until some specific timestamp in the future, rather than for a specified duration. To work around it you would need to implement your own version of Thread.sleep() that uses System.nanoTime() instead of System.currentTimeMillis() or any other time-dependent API. How to do that without using the built-in Thread.sleep() I can't say, however.

Edit:

Or, what if you create some external app in another language (like C or whatever else you prefer) that does nothing but wait for a specified duration and then exit. Then instead of calling Thread.sleep() in Java, you can spawn a new instance of this external process, and then call waitFor() on it. This will "sleep" the Java thread for all practical purposes, and so long as your external app is able to sleep for the correct duration, it will resume at the correct time without getting frozen and without thrashing the CPU.

Seems like a long way to go to fix the issue, but it's the only feasible workaround that I can think of. Also, given that spawning an external process is a relatively expensive operation, it probably works best if you are sleeping for a relatively long time (like several hundred ms or more). For shorter durations it might just continue thrashing the CPU.

aroth
  • 54,026
  • 20
  • 135
  • 176
  • Actually, I have seen two different behaviors on two machines. On the first machine, the thread awakes when the time catches up. On the other one, the thread stays asleep. In the ticket, the second case is described. – barjak May 06 '11 at 10:01
  • About the edit : I tried this approach, and it works fine. I'll try to investigate some other solutions based on the same principle (using an external process or service) – barjak May 06 '11 at 11:19
  • @barjak - Very cool. I'd be interested in knowing how you coded it, and also what the exact overhead is in launching the external process, if you took the time to measure/quantify it. – aroth May 06 '11 at 11:36
  • Finally, I don't use a monitoring thread anymore (see the epilogue of my question). The code was as simple as what you described : `Runtime.exec("sleep 0.2").waitFor()`. The resulting sleep is quite precise, but it annoys me to run a process each 200ms... – barjak May 11 '11 at 16:13
3

As others have said, you definitely shouldn't have to change the system clock. The timestamp (milliseconds since the epoch) is consistent across all computers across the world, but the local time depends on your location, observation on Daylight Savings Time and so on. Therefore, the problem is with the OS locale and time/date settings.

(Still, I agree that if the system clock does change, the JVM should detect this and update or awaken sleeping threads to combat the problem.)

aroth
  • 54,026
  • 20
  • 135
  • 176
BoffinBrain
  • 6,337
  • 6
  • 33
  • 59
  • I added a note at the end of my question to clarify this point. – barjak May 06 '11 at 10:20
  • Oh, I see the problem now. That's nasty... Your 'warning system' seems reasonable in such cases. I was going to suggest keeping a reference of all `Thread`s and waking them in the event of a clock change, but not only will it not be possible to get a reference to them all, but waking them unconditionally would probably have bad side-effects. – BoffinBrain May 06 '11 at 10:51
  • `Thread.enumerate` can give you the reference of all threads. To force their wake up, it's more difficult. I could call `interrupt` on the threads, but the convention is to use this signal as an instruction to cleanly terminate the thread, not only to wake up the sleeping threads. – barjak May 06 '11 at 11:12
  • One workaround you could use is to create utility methods which wrap around `Thread.sleep()` and other calls you need to make, while holding the value of the system milliseconds value before sleeping. The method would catch `InterruptedException` and then check the system time to see again to see if they've been woken too early, and if so, sleep again (recursively or iteratively). If not, `InterruptedException` will get thrown as normal, preserving standard behaviour. – BoffinBrain May 06 '11 at 12:33
  • I could do that, but that would work only for my own code. Any third party code will interpret `interrupt` as a signal to stop the thread. – barjak May 06 '11 at 13:03
  • Damn. Do you know if other JVMs suffer from the same issue, and do you have the ability to control what JVM is used on the client machine (perhaps by bundling it as part of an installer)? – BoffinBrain May 06 '11 at 14:15
  • I tried on 3 machines only, each with a different Java 6 VM : a 64 bit OpenJDK on an archlinux, a 32 bit Hotspot on an ubuntu, and a 32 bit hotspot on a windows (each with the same issue). Our code needs a java 6 VM. And yes, we can bundle a JVM with the soft, but which one... – barjak May 06 '11 at 14:48
  • If all of the JVMs suffer the same problem (and that's really awful on their part - how long has Java been around now?) then the only other thing I can think of is hacking the JDK itself. I suggested the utility method for interruptible waiting above, but as you say, 3rd-party APIs will still suffer. However, if you modify `Object` directly to override the native `wait` method, and `Thread` to override `sleep` then you might get around it in a very hacky fashion. – BoffinBrain May 06 '11 at 15:00
2

Please test the latest jre 1.7.0_60. It resolves the described problem caused by a system time shift to the past at least for systems with a glibc version released since 2009.

Related bug http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6900441 has been fixed and therefore all functions mentioned by you (Thread.sleep, Object.wait, Thread.join, LockSupport.parkNanos, java.util.Timer and java.swing.Timer) should work as expected.

I have tested it with a linux kernel 3.7.10.

Uli
  • 1,390
  • 1
  • 14
  • 28
0

@Op. You have implemented something that looks like "busy waiting", and that will always consume lots of resources.

I agree with the others, I don't see why you need to change the system clock when you go from summer to winter time.

Kaj
  • 10,862
  • 2
  • 33
  • 27
  • 1
    I have no other choice than to do a busy wait, for the monitoring thread : if I sleep in the infinite loop, the monitoring thread will freeze too ! – barjak May 06 '11 at 09:54
0

You don't change OS time for DST adjustments! It's nothing more than a Time Zone change. System clock should always be GMT. And the wall clock time that you display to the user is derived from that with the proper time zone offset.