How do I explain Insert performance for HashMap in Java 8 and Java 7

Question

I was going through Java 8 HashMap improvements which shows search performance improvement by around 20 % using Binary tree.

At that point there was a thought how is the insert performance impacted. So I started with inserting few millions of records. Below is the code snippet and different set of results.

import java.util.HashMap;
import java.util.Map;

public class MapWriter {

public static final int MAX_KEY = 1_000_000;

private Map<Double, Double> map = new HashMap<>(MAX_KEY);


public static void main(String[] args) {
    long startTime = System.currentTimeMillis();

    MapWriter writer = new MapWriter();

    for (int i = 0; i < MAX_KEY; i++) {
        double random = Math.random();
        writer.map.put(random, random);
    }

    long timeTaken = System.currentTimeMillis() - startTime;

    System.out.println("Total Time Taken = " + timeTaken);
    System.out.println("Map Size = " + writer.map.size());
}

}

Here are different results:

1. 10 Million Insertions

   • Java 7

     • Total Time Taken = 23145

   • Java 8

     • Total Time Taken = 64964

2. 2 Million Insertions

   • Java 7
     • Total Time Taken = 6628
   • Java 8
     • Total Time Taken = 8312

3. 1 Million Insertions

   • Java 7
     • Total Time Taken = 3577
   • Java 8
     • Total Time Taken = 1212

The result shows that up to 1 million insertions Java 8 performs better. But the result shows opposite behavior as you move upwards.

How do I explain this behavior??????

Update: Thank you guys for your precious feedback. I need to learn more on benchmarking. I pre-initialized Math.random part and the results were same for both java 7 and java 8. Here is the modified code. Please let me know if the code still looks smelly from benchmarking perspective.

public class MapWriter {

public static int MAX_KEY = 1_000_000;


private Map<Double, Double> map = new HashMap<>(MAX_KEY);


public static void main(String[] args) {
    MAX_KEY = Integer.parseInt(args[0]);

    Double[] keys = new Double[MAX_KEY];

    for(int i = 0; i < MAX_KEY; i++) {
        keys[i] = Math.random();
    }

    MapWriter writer = new MapWriter();
    for (int i = 0; i < 100000; i++) {

        writer.map.put(keys[i], keys[i]);
    }

    writer.map = new HashMap<>(MAX_KEY);
    long startTime = System.nanoTime();


    for (int i = 0; i < MAX_KEY; i++) {
        double random = Math.random();
        writer.map.put(random, random);
    }

    long timeTaken = System.nanoTime() - startTime;

    System.out.println("Total Time Taken = " + timeTaken / 1000000);
    System.out.println("Map Size = " + writer.map.size());
}

}

Answer 1

If you write your own micro-benchmark instead of using a dedicated tool like JMH you should consider some important points:

• Use System.nanoTime(). for measuring elapsed time to be immune against time changes triggered outside the JVM.
• Don’t use Math.random() (unless you want to benchmark Math.random()). Using a global resource like the global random generator adds thread synchronization that might affect your code.
• Use a Random instance as a source for your random inputs, but initialize it with a constant seed to ensure that the runs you want to compare really do the same

• run the code you want to benchmark multiple times within the JVM to ensure you don’t measure initialization overhead like class loading, first time memory allocations, interpreted code, etc.

• Generally, it’s a good idea to run a profiler during test runs to verify that the time is really spent within the code you want to benchmark

Using the last bullet above, the main problem with your benchmark is revealed very fast. You didn’t give the JVM enough initial memory (and maybe even the maximum memory is too limited) and ran the code only one time. So you are measuring mainly impacts of the memory management here.

Note that the Java 8 HashMap requires slightly more memory and when measuring the initialization cost only, even slightly more memory requirements may have a big impact .

Giving both JVMs at least 1GB initial memory for 10 Million Insertions resulted in taking about 5 seconds on Java 8 and 6 seconds on Java 7 on my machine, even without warmup. Far away from your 20 seconds not to speak of more than a minute.

The bottom line is that you need far more runs with different environmental parameters before even formulating an assumption about the reason for the different results. When you ran the test on a 32 Bit JVM, -server and -client, a 64Bit JVM, all of them with different memory settings and get consistent results for all of them showing a particular version being faster or slower, then you might suggest that it is the version. But there might still be other reasons…

Answer 2

I think you're also testing Math.random() and auto-boxing while you run the benchmark. You have to remove them outside the benchmark scope. Furthermore I think you have to re-run your test several times and take an average instead of a single run.