Benchmark Java Method Calls

Question

I try to benchmark java method calls. The problem with those calls is that they are getting inlined and optimized.

So in the end I try to look for a way to avoid those optimizations. Currently I use:

for(int index = 0; index < 100_000_000; index++)
    value = value * 2 % 8;

This calculation is replaced by methods for JNI and Java. The output of the benchmark is:

Druation static jni = 1002ms, instance jni = 1000ms, static java = 136ms, instance java = 140ms, raw = 145ms

Druation static jni = 975ms, instance jni = 974ms, static java = 132ms, instance java = 128ms, raw = 134ms

Druation static jni = 966ms, instance jni = 1084ms, static java = 127ms, instance java = 130ms, raw = 135ms

Druation static jni = 958ms, instance jni = 1083ms, static java = 127ms, instance java = 131ms, raw = 134ms

Druation static jni = 957ms, instance jni = 1085ms, static java = 126ms, instance java = 131ms, raw = 135ms

From the Java vs raw (no methods) one sees the affect of inlining. I would like to know if there is something I can do to prevent inlining?

Cause I mix C and Java in a single Class showing that I have control in a certain limit. — Martin Kersten, Apr 18 '15 at 07:52
Dasiabling optimizations? That should never be done for performance benchmarks. A program might be slower than another without optimizations, but faster with optimization because it's structure allows it to be better optimized than the other program. Use a proper framework for benchmarks. — AliciaBytes, Apr 18 '15 at 07:53
@Raphael Well I need to benchmark without inlining. I just want a simple measure what an unoptimized java call really costs and how it compares to a JNI call. After I learned that JMH can disable inlining I am even more curious how this is achieved. — Martin Kersten, Apr 18 '15 at 08:01

score 0 · Answer 1 · edited May 23 '17 at 11:58

Disclaimer

Modern JVMs are too complex, and do all kinds of optimization. If you try to measure some small piece of code, it is really complicated to do it correctly without very, very detailed knowledge of what the JVM is doing. In order to work around pitfalls common to HotSpot/OpenJDK it is important to use JMH. Also it is important to know how to write a correct microbenchmark

Answer A

In order to avoid inlining JMH uses compiler commands: -XX:CompileCommand=command,method[,option]

for example:

-XX:CompileCommand=dontinline,java.lang.String::indexOf

Note: Compile command might get ignored by the compiler. Please check compiler logs and generated code:

-XX:+UnlockDiagnosticVMOptions
-XX:+PrintCompilation
-XX:+LogCompilation
-XX:LogFile=./compiler.log
-XX:+PrintInlining

Answer B

The most reliable way is to disable inlining at all -XX:-Inline

Answer C

The best way is to use JMH in this way.

Answer D

The longest way is to wait until JEP 165: Compiler Control is finished.