Java String = "" vs. new String("") performance change

Question

I have made the same Test as it was done in this post: new String() vs literal string performance

Meaning I wanted to test which one has the better in performance. As I expected the result was that the assignment by literals is faster. I don't know why, but I did the test with some more assignments and I noticed something strange: when I let the program do the loops more than 10.000 times the assignment by literals is relatively not as much faster than at less than 10.000 assignments. And at 1.000.000 repetitions it is even slower than creating new objects.

Here is my code:

double tx = System.nanoTime();
for (int i = 0; i<1; i++){
    String s = "test";
}
double ty = System.nanoTime();

double ta = System.nanoTime();
for (int i = 0; i<1; i++){
    String s = new String("test");
}
double tb = System.nanoTime();

System.out.println((ty-tx));
System.out.println((tb-ta));

I let this run like it is written above. I'm just learning Java and my boss asked me to do the test and after I presented the outcome of the test he asked me do find an answer, why this happens. I cannot find anything on google or in stackoverflow and so I hope someone can help me out here.

factor at         1 repetition   3,811565221
factor at        10 repetitions  4,393570401
factor at       100 repetitions  5,234779103
factor at     1,000 repetitions  7,909884116
factor at    10,000 repetitions  9,395538811
factor at   100,000 repetitions  2,355514697
factor at 1,000,000 repetitions  0,734826755

Thank you!

Answer 1

First you'll have to learn a lot about the internals of HotSpot, in particular the fact that your code is first interpreted, then at a certain point compiled into native code.

A lot of optimizations happen while compiling, based on results of both static and dynamic analysis of your code.

Specifically, in your code,

String s = "test";

is a clear no-op. The compiler will emit no code whatsoever for this line. All that remains is the loop itself, and the whole loop may be eliminated if HotSpot proves it has no observable outside effects.

Second, even the code

String s = new String("test");

may result in almost the same thing as above because it is very easy to prove that your new String is an instance which cannot escape from the method where it is created.

With your code, the measurements are mixing up the performance of interpreted bytecode, the delay it takes to compile the code and swap it in by On-Stack Replacement, and then the performance of the native code.

Basically, the measurements you are making are measuring everything but the effect you have set out to measure.

To make the arguments more solid, I have repeated the test with jmh:

@OutputTimeUnit(TimeUnit.NANOSECONDS)
@BenchmarkMode(Mode.AverageTime)
@Warmup(iterations = 1, time = 1)
@Measurement(iterations = 3, time = 1)
@Threads(1)
@Fork(2)
public class Strings
{
  static final int ITERS = 1_000;
  @GenerateMicroBenchmark
  public void literal() {
    for (int i = 0; i < ITERS; i++) { String s = "test"; }
  }

  @GenerateMicroBenchmark
  public void newString() {
    for (int i = 0; i < ITERS; i++) { String s = new String("test"); }
  }
}

and these are the results:

Benchmark     Mode   Samples         Mean   Mean error    Units
literal       avgt         6        0.625        0.023    ns/op
newString     avgt         6       43.778        3.283    ns/op

You can see that the whole method body is eliminated in the case of the string literal, while with new String the loop remains, but nothing in it because the time per loop iteration is just 0.04 nanoseconds. Definitely no String instances are allocated.