Ensuring JMH benchmark method gets compiled

Question

I have a benchmark with a @Param controlling the input size, such as:

@State(Scope.Benchmark)
class MyBenchmark {
  @Param({"100", "10000", "1000000"})
  public int ARRAY_SIZE;
  public int[] INPUT;

  @Setup
  public void setUp() {
    INPUT = new int[ARRAY_SIZE];
    // ...
  }

  @Benchmark
  public void compute() {
    // ...
  }

}

When the input is large, compute doesn't get invoked enough times to trigger compilation during warmup. Since I would like to measure peak performance, I want to ensure the method is invoked enough during warmup to be compiled.

Is there a good way to do this? I could set a higher number of warmup iterations, but this would apply to all input sizes when it's really only necessary for the large ones which don't get compiled.

score 1 · Accepted Answer · answered Jun 05 '22 at 14:05

Instead of increasing the number of iterations, you can try making compilation happen earlier.

Since JDK 9, there is -XX:CompileThresholdScaling JVM option to adjust frequencies and thresholds of the compilation policy. The lower is the value - the earlier compilation starts.
E.g. -XX:CompileThresholdScaling=0.05 will make thresholds 20x lower.

The option can be applied globally or on a per-method basis. For example, to apply it only to compute benchmark, add the following annotation:

@Benchmark
@Fork(jvmArgsAppend = {"-XX:CompileCommand=CompileThresholdScaling,*MyBenchmark*::compute*,0.05"})
public void compute() {
  // ...
}

Alternatively, you may extract larger parameter values into another @State object and create a separate benchmark method with different @Warmup options:

@State(Scope.Benchmark)
public static class SmallParams {
    @Param({"100", "10000"})
    int size;
}

@State(Scope.Benchmark)
public static class LargeParams {
    @Param({"1000000"})
    int size;
}

@Benchmark
@Warmup(iterations = 5)
public void computeSmall(SmallParams params) {
    //
}

@Benchmark
@Warmup(iterations = 50)
public void computeLarge(LargeParams params) {
    //
}

Or just run the benchmark twice, overriding parameters in the command line:

-p size=100,10000 -wi 5
-p size=1000000 -wi 50

Ensuring JMH benchmark method gets compiled

1 Answers1