4

Considering the default usage of the JMH, I would like to make sure which type of time JMH bases its measurements on: CPU time or Wall-clock.

I tried looking into the JMH official samples (https://openjdk.java.net/projects/code-tools/jmh/), tutorials (at Jenkov, Baeldung, Mykong and others), and did not manage to find this information precisely (I acknowledge I might have missed some documentation or general information on benchmarks).

For instance, in Sample 35 (https://hg.openjdk.java.net/code-tools/jmh/file/99d7b73cf1e3/jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_35_Profilers.java), we have as an example of output: 

        Benchmark                              (type)  Mode  Cnt     Score    Error   Units

          JMHSample_35_Profilers.Maps.test     hashmap  avgt    5  1553.201 ±   6.199   ns/op

          JMHSample_35_Profilers.Maps.test     treemap  avgt    5  5177.065 ± 361.278   ns/op

Hence, I would like to know if the Score column was calculated using Wall-clock time or CPU Time so that I can accurately interpret the benchmark results.

rfguimaraes
  • 41
  • 4

1 Answers1

0

As I feared, I had missed basic information and formulated my question badly since wall-clock and CPU times are not the only options. I followed Alexandre Cartapanis' suggestion and looked into JMH's source code. I found out that the class BenchmarkGenerator, which produces the benchmarks, relies on System.nanoTime().

Further, according to System.nanoTime() documentation and Kevin Bourrillion's answer to another question I concluded that JMH, by default, measures time according to some arbitrary, but fixed, source (just like System.nanoTime()).

rfguimaraes
  • 41
  • 4