23

I saw on the Internet that I was supposed to use System.nanoTime() but that doesn't work for me - it gives me the time with milliseconds precision. I just need the microseconds before and after my function executes so that I know how long it takes. I'm using Windows XP.

Basically, I have this code that, for example, does 1 million up to 10 millions of insertions in a java linked list. The problem is that I can't measure the precision right; sometimes it takes less time to insert everything in the smaller list.

Here's an example:

class test
{
    public static void main(String args[])
    {
        for(int k=1000000; k<=10000000; k+=1000000)
        {
            System.out.println(k);
            LinkedList<Integer> aux = new LinkedList<Integer>();
            //need something here to see the start time
            for(int i=0; i<k; i++)
                aux.addFirst(10000);
            //need something here to see the end time
            //print here the difference between both times
        }
    }
}

I did this many times - there was an exterior loop doing it 20 times for each k - but the result aren't good. Sometimes it takes less time to to make 10 million insertions than 1 million, because I'm not getting the correct measured time with what I'm using now (System.nanoTime())

Edit 2: Yes, I'm using the Sun JVM.

Edit 3: I may have done something wrong in the code, I'll see if changing it does what I want.

Edit 4: My mistake, it seems System.nanoTime() works. Phew.

Hearen
  • 7,420
  • 4
  • 53
  • 63
  • What version of Java? On my XP machine w/ JDK1.6.0_06, nanoTime() has precision through the last decimal places. I'm assuming your version returns values that always end in three zeros. – basszero Feb 02 '09 at 17:04
  • Sun JVM is not enough, which VERSION of it? (Your answer was correct, I didnt' ask the right question. I assum everyone uses the Sun JVM) – basszero Feb 02 '09 at 17:10
  • Java -version in the cmd gives me: java version "1.7.0-ea" And another 2 lines. Either way, it was a mistake in my code. Sorry, I couldn't find the error even after a while searching for it. –  Feb 02 '09 at 17:12
  • brave soul, 1.7 is still WAY early – basszero Feb 02 '09 at 17:32
  • Adding comment since it's OT. You should read up on how to write micro benchmarks for the JVM, ex http://www.concentric.net/~ttwang/tech/microbench.htm Also, you probably want to box that 10000 integer outside the loop. – John Nilsson Feb 02 '09 at 17:34
  • 8
    A benchmark that requires microsecond precision isn't going to be very accurate. You can add noise a microsecond time measurement with the slightest extra CPU load- if another machine on your network happens to ping your box, if you move your mouse, etc. – Kip Feb 02 '09 at 18:37
  • This doesn't answer your specific question, but using this might make it unnecessary: http://code.google.com/p/caliper/ – TJR Oct 27 '11 at 20:54

12 Answers12

35

My guess is that since System.nanoTime() uses the "most precise available system timer" which apparently only has millisecond-precision on your system, you can't get anything better.

Zach Scrivena
  • 29,073
  • 11
  • 63
  • 73
25

It's not clear to me exactly what you're benchmarking, but in general, any test which takes such a short amount of time to run, that accuracy lower than 50 ms is relevant, is going to be very prone to other disturbances.

I generally try to make benchmarks run for at least 10 seconds. The framework I'm writing at the moment will guess how many iterations to run so that it will take 30 seconds. That means you won't get radically different results just because some other process stole the CPU for a few milliseconds.

Running for longer is almost always a better approach than trying to measure with finer-grained accuracy.

Hearen
  • 7,420
  • 4
  • 53
  • 63
Jon Skeet
  • 1,421,763
  • 867
  • 9,128
  • 9,194
8

Using java.time

FYI, Java 9 and later has a fresh implementation of Clock that can capture the current moment in up to nanoseconds resolution.

The Instant class represents a moment on the timeline in UTC with a resolution of nanoseconds (up to nine (9) digits of a decimal fraction).

Call Instant.now to capture the current moment.

  • In Java 9 and later, you get a resolution up to nanoseconds for that current moment.
  • In Java 8, the current moment is captured only up to milliseconds resolution (you can indeed hold values with nanoseconds, but only capture current moment in milliseconds).

Code:

Instant instant = Instant.now() ;

Represent a span of time unattached to the timeline with Duration class. Holds an amount of time in terms of seconds and nanoseconds.

Duration d = Duration.between( instantThen , Instant.now() );

To be clear, the microseconds resolution asked in the Question is in between the granularities of milliseconds and nanoseconds. Number of places in a decimal fraction: millis is 3 (0.123), micros is 6 (0.123456), nanos is 9 (0.123456789).

Caveat

Java relies on your computer’s hardware clock. As others warned, that hardware will almost certainly capture time with far less accuracy and far less resolution than nanoseconds.

Benchmarking at such fine granularity is fraught with issues and not advisable generally.

And beware of premature optimization.

There is a proposal to add micro-benchmarking facility to the Java platform in JEP 230: Microbenchmark Suite. Based on the Java Microbenchmark Harness (JMH).

About java.time

The java.time framework is built into Java 8 and later. These classes supplant the troublesome old legacy date-time classes such as java.util.Date, Calendar, & SimpleDateFormat.

The Joda-Time project, now in maintenance mode, advises migration to the java.time classes.

To learn more, see the Oracle Tutorial. And search Stack Overflow for many examples and explanations. Specification is JSR 310.

Where to obtain the java.time classes?

The ThreeTen-Extra project extends java.time with additional classes. This project is a proving ground for possible future additions to java.time. You may find some useful classes here such as Interval, YearWeek, YearQuarter, and more.

Basil Bourque
  • 303,325
  • 100
  • 852
  • 1,154
4

System.nanoTime() uses a counter in the CPU and is usually accurate to about 1 micro-second on Windows XP and Linux.

Note: Windows XP is often less accurate on multi-cpu machines as it doesn't compensate for different CPUs having different counters. Linux does. Note 2: It will drift relative to the System.currentTimeMillis() as it is based on the accuracy of the clock for your CPU (which doesn't need to be so accurate over a period of time), rather than the clock you have for getting the time.(which drifts less per day, but has less granularity)

In your benchmark you are basically testing the speed at which you can create new objects. Not surprisingly your results will vary dramatically based on your GC settings and how recently a GC has been performed.

Try running your tests with the following options and you should see very different results.

-verbosegc -XX:NewSize=128m -mx256m
Hearen
  • 7,420
  • 4
  • 53
  • 63
Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130
3

That's weird. System.nanoTime() is supposed to work. Are you using the Sun JVM?

Can you just repeat your operation 1000 times and divide the time by 1000 to find out what you need to know?

Sarel Botha
  • 12,419
  • 7
  • 54
  • 59
3

You have to repeat the tests thousands of times. There are lots of things happening that will influence your measurements, like garbage collection, I/O, swap in/out, the size of the ready-queue threads, etc.

Hearen
  • 7,420
  • 4
  • 53
  • 63
Tiago
  • 9,457
  • 5
  • 39
  • 35
3

If you want a reliable result, use a profiler. I suggest VisualVM, which is easy to install and is bundled with the JDK starting from version 1.6.0_07.

It is an easy to use visual tool that integrates several commandline JDK tools and lightweight profiling capabilities.

Hearen
  • 7,420
  • 4
  • 53
  • 63
marcospereira
  • 12,045
  • 3
  • 46
  • 52
1

It may be the case that the underlying OS doesn't provide timers with nanosecond precision.

There is also an older post.

Community
  • 1
  • 1
starblue
  • 55,348
  • 14
  • 97
  • 151
1

Yes, the accuracy and precision of System.nanoTime is usually much better than System.currentTimeMillis, but without guarantee: it can become just as bad in the worst case.

ThreadMXBean.getCurrentThreadCpuTime tends to yield smaller times, but its resolution is unclear, and it has further disadvantages (do you really want the CPU time?, platform dependent semantics, supported on your platform?).

Measuring the time with all three techniques also has some cost, i.e. requires time itself, which can distort the measurements. The costs are highly platform dependent, but often cost(System.currentTimeMillis) << cost(System.nanoTime) << cost(ThreadMXBean.getCurrentThreadCpuTime).

About micro benchmarking in general, see

Community
  • 1
  • 1
DaveFar
  • 7,078
  • 4
  • 50
  • 90
0

a "quick and dirty" solution that I eventually went with:

TimeUnit.NANOSECONDS.toMicros(System.nanoTime());

UPDATE:

I originally went with System.nanoTime but then I found out it should only be used for elapsed time, I eventually changed my code to work with milliseconds or at some places use:

TimeUnit.MILLISECONDS.toMicros(System.currentTimeMillis());

but this will just add zeros at the end of the value (micros = millis * 1000)

Left this answer here as a "warning sign" in case someone else thinks of nanoTime :)

keisar
  • 5,266
  • 5
  • 28
  • 28
0

For our recent profiling, I found that ThreadMXBean.getCurrentThreadCpuTime() and the option -XX:+UseLinuxPosixThreadCPUClocks did what we needed.

See http://bugs.java.com/view_bug.do?bug_id=6888526 for more details

Hearen
  • 7,420
  • 4
  • 53
  • 63
Andrew Jessop
  • 135
  • 2
  • 7
0

Such a benchmark that relies on short time-interval gives you unreliable results. You will always get different results, because of external factors like I/O, Swapping, Process Switches, Caches, Garbage Collection etc. Additionally the JVM optimizes your calls, so it's likely that the first measured things are going slower than later call. The JVM starts more and more to optimize the commands you execute.

Additionally the method like System.nanoTime() is dependent on the timers of the underlying system. They may (and most likely will) not have the granularity to measure in that accuracy. To cite the API:

This method provides nanosecond precision, but not necessarily nanosecond accuracy. No guarantees are made about how frequently values change.

To really measure with high precision you need to access an external timing hardware with guaranteed precision.

To make your benchmark more stable you need to execute it more than once and to measure bigger time-intervals than only milliseconds.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Mnementh
  • 50,487
  • 48
  • 148
  • 202