Java performance issues

Question

I'm in the process of benchmarking an app i've written. I ran my app through the benchmark 10 times in a loop (to get 10 results instead of only 1). Each time, the first iteration seems to take some 50 - 100 milliseconds longer than rest of the iterations.

Is this related to the JIT compiler and is there anything one could do to "reset" the state so that you would get the initial "lag" included with all iterations?

Think of it as [turbo lag](http://en.wikipedia.org/wiki/Turbocharger#Turbo_lag). — Joachim Sauer, Feb 18 '11 at 13:57
You might be interested in ["How do I write a correct micro-benchmark in Java"](http://stackoverflow.com/questions/504103/how-do-i-write-a-correct-micro-benchmark-in-java). — Joachim Sauer, Feb 18 '11 at 13:58
@Joachim Sauer: Good stuff, good read. I'd +1 you if it be possible through comments :) — JHollanti, Feb 19 '11 at 13:00

score 3 · Answer 1 · answered Feb 18 '11 at 14:02

To benchmark a long running application you should allow an initialization (1st pass), thats because classes have to be loaded, code has to be generated, in web-apps JSP compile to servlets etc. JIT of course plays its role also. Sometimes a pass could take longer if garbage collection occurs.

score 1 · Accepted Answer · answered Feb 18 '11 at 13:58

1

It is probably caused by the JIT kicking in, however you probably want to ignore the initial lag anyway. At least most benchmarks try to, because it heavily distorts the statistics.

You can't "uncompile" code that has been compiled but you can turn compiling off completely by using the -Xint command line switch.

answered Feb 18 '11 at 13:58

biziclop

48,926
12
77
104

*You can't "uncompile"*... Actually, Hotspot often deoptimizes compiled code (and ultimately re-compiles again). For instance it may deoptimize some code if new class is loaded (same hierarchy for example). – bestsss Feb 25 '11 at 23:36
@bestsss Fair enough but you have no way of controlling this behaviour. So in a way it's not you doing it, it's just Hotspot. :) – biziclop Feb 25 '11 at 23:48
I mentioned loading classes: you can generate some classes (on the fly) and force stuff to happen, but heck. Make well sure you know what you do and why. Most people coming to this site should not even see that remark for I will be battered :D – bestsss Feb 25 '11 at 23:50
@bestsss Don't worry, most people won't understand it anyway. :) But it's a good idea, all you need is to create a new classloader for each run. – biziclop Feb 25 '11 at 23:56
might need to call some code to fall in the interpreter part and what not. It's really knowing how hotspot exactly optimizes. I am no JVM engineer though, even if I have far above the average idea how JVM works under the hood. – bestsss Feb 26 '11 at 00:05

score 1 · Answer 3 · answered Feb 18 '11 at 14:24

1

The first pass will probably always be slower because of the JIT. I'd even expect to see differences when more runs are made because of possible incremental compilation or better branch prediction.

For benchmarking, follow the recommondations given in the other answers (except I wouldn't turn off the JIT because you'd have your app running with JIT in a production environment).

In any case use a profiler such as JVisualVM (included in JDK).

answered Feb 18 '11 at 14:24

Axel

13,939
5
50
79

To be fair, I wouldn't turn off the JIT either, except to establish whether the lag was indeed caused by it. – biziclop Feb 18 '11 at 14:33

Pantelis Sopasakis · Answer 4 · 2011-02-18T14:35:32.963

0

There are certain structures you might have in your code, such as singletons which are initialized only once and consume system resources. If you're using a database connection pool for example, this might be the case. Moreover it is the time needed by Java classes to be initialized. For these reasons, I think you should discard that first value and keep only the rest.

edited Feb 18 '11 at 14:35

answered Feb 18 '11 at 14:03

Pantelis Sopasakis

1,902
5
26
45

JIT compilation happens at runtime and *does* attempt to speed things up. – Jeremy Feb 18 '11 at 14:06
@Jeremy Heiler You are right about just-in-time compilation (I updated my answer). However I think initialization of objects in memory usually plays an important role. – Pantelis Sopasakis Feb 18 '11 at 14:36

Stephen C · Answer 5 · 2011-02-18T14:34:57.627

Is this related to the JIT compiler

Probably yes, though there are other potential sources of "lag":

Bootstrapping the JVM and creation of the initial classloader.
Reading and loading the application's classes, and the library classes that are used.
Initializing the classes.
JIT compilation.
Heap warmup effects; e.g. the overheads of having a heap that is initially too small. (This can result on the GC running more often than normal ... until the heap reaches a size that matches the application's peak working set size.)
Virtual memory warmup effects; e.g. the OS overheads incurred when the JVM grows the process address space and physical pages are allocated.

... and is there anything one could do to "reset" the state so that you would get the initial "lag" included with all iterations?

There is nothing you can do, apart from starting the JVM over again.

However, there are things that you can do to remove some of these sources of "lag"; e.g. turning of JIT compilation, using a large initial heap size, and running on an otherwise idle machine.

Also, the link that @Joachim contributed above is worth a thorough read.

Java performance issues

5 Answers5