If I am designing a test on sorting algorithm can I do this way to avoid JVM warmup ? Thank you!
double count = 0;
double start, end;
for(int r = 0; r < warmup; r++) {
// do test
}
for(int t = 0; t < runs; t++){
start = System.nanoTime();
// do test
end = System.nanoTime();
count += start - end;
}
double avg = count/avg
The JVM warmup usually refers to the time it takes for the JVM to find the hotspots and JIT these sections of the code. If you run your actual tests a few hundred (actually a few thousand I believe) times you should be fairly good to go.
You should however know that, even if you do this, there are no guarantees. You'll have to experiment with your particular JVM to figure out how much work you have to do before the vital parts is JITed and so on.
In this little case study the JIT compilation kicked in after 1700 calls.
If I am designing a test on sorting algorithm can I do this way to avoid JVM warmup ?
Some pedantry first. You should not avoid JVM warmup. It needs to happen. What you are trying to do is to prevent the JVM warmup from distorting your benchmark results.
To answer your question, the approach is roughly right, but it is very difficult to predict how many times you need to do the test in the initial loop. It is likely to depend on the test code, on the JVM version and on JVM tuning parameter ... and probably other things besides.
What I normally do is to just print the raw timings, filter out the initial "warmup" iterations that appear to have anomalous timing values "by eye", and then calculate the averages by hand. It is clunky, but it gives me some confidence that I've accounted for warmup and other possible sources of anomalies.
This is a very big area, but here's a couple of tips:
1) ensure your FULL test (including the iteration loop) is in a subroutine that is repeatedly invoked. So your test has the for() loop in the "parent" method. push it down to a "child" and invoke that repeatedly. This allows the various JIT technologies to really do a full optimization without having to do in-flight code replacement (dynamic loop transfer, etc..)
2) Ensure the test runs for a long time AFTER a long warmup. 30s is bare minimum for the real measurement period, after an equally long warmup, if possible. For example, SPECjbb, etc.. run for several minutes per iteration, for multiple iterations.
Yes. Since the warmup loop runs the actual test, it means all classes etc will have been loaded and JIT compilations should have been run.
