Encryption and decryption in loop gives faster result after first iteration in Java

Question

I am testing a case where a 32 bytes data is encrypted multiple times with different public keys using RSA. Here is my code:

    List<Long> colDurSetup = new ArrayList();
    List<Long> colDurEnc = new ArrayList();
    List<Long> colDurDec = new ArrayList();
    List<PublicKey> pubKeys = new ArrayList();
    List<PrivateKey> privKeys = new ArrayList();

    for(int i=0; i<10;i++) {
        long timeStart = System.currentTimeMillis();
        KeyPair kp = buildKeyPair();
        long timeEnd = System.currentTimeMillis();
        long dur = timeEnd-timeStart;
        colDurSetup.add(dur);
        pubKeys.add(kp.getPublic());
        System.out.println(kp.getPublic().getEncoded().length);
        privKeys.add(kp.getPrivate());
        System.out.println(kp.getPrivate().getEncoded().length);
    }

    SecureRandom r = new SecureRandom();
    byte[] data = new byte[32];
    r.nextBytes(data);
    System.err.println(Arrays.toString(data));

    System.out.println("data length "+data.length);

    for(int i=0;i<10;i++) {
        long timeStart = System.nanoTime();
        byte[] ciphertext = encrypt(pubKeys.get(i), data);
        long timeEnd = System.nanoTime();
        long dur = timeEnd-timeStart;
        System.out.println("enc duration "+dur);
        colDurEnc.add(dur);
        System.out.println(timeStart);
        System.out.println(timeEnd);

        System.out.println("ciphertext length "+ciphertext.length);
        System.err.println("cip "+Arrays.toString(ciphertext));

        long timeStart2 = System.nanoTime();
        byte[] decrypted = decrypt(privKeys.get(i), ciphertext);
        long timeEnd2 = System.nanoTime();

        long dur2 = timeEnd2-timeStart2;
        colDurDec.add(dur2);
        System.err.println("dec "+Arrays.toString(decrypted));
        System.out.println("dec duration "+dur2);
    }

public static KeyPair buildKeyPair() throws NoSuchAlgorithmException {
    final int keySize = 2048;
    KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance("RSA");
    keyPairGenerator.initialize(keySize, random);      
    return keyPairGenerator.genKeyPair();
}

public static byte[] encrypt(PublicKey publicKey, byte[] message) throws Exception {
    Cipher cipher = Cipher.getInstance("RSA");  
    cipher.init(Cipher.ENCRYPT_MODE, publicKey, random);  
    return cipher.doFinal(message);  
}

public static byte[] decrypt(PrivateKey privateKey, byte [] encrypted) throws Exception {
    Cipher cipher = Cipher.getInstance("RSA");  
    cipher.init(Cipher.DECRYPT_MODE, privateKey);
    return cipher.doFinal(encrypted);
}

When I run it, I noticed that the first iteration requires more time to encrypt and decrypt than the rest of iterations and I'm not sure why. Here is the result:

enc duration 231
dec duration 4
enc duration 0
dec duration 4
enc duration 0
dec duration 4
enc duration 0
dec duration 4
enc duration 0
dec duration 4
enc duration 1
dec duration 3
enc duration 0
dec duration 4
enc duration 0
dec duration 4
enc duration 0
dec duration 4
enc duration 0
dec duration 4

I assumme because it runs the same then all iteration should give the same duration but turns out I was wrong. Is there anything wrong in the code?