How to decide which execution time is more reputable while using System.nanoTime ()

Question

When i try to measure the performance execution speed time of my method below:

  public static int FFMulFast(int firstPol, int secondPol){
  int temp = 0;

  if (firstPol == 0 ||secondPol == 0)

  return 0;

 /*The multiplication is done by using lookup tables. We have used both logarithmic and exponential table for mul
 the idea is firstly look to Logarithmic table then add their powers and find the corresponding of this to exponential table */

   temp = (Log[(firstPol & 0xff)] & 0xff) + (Log[(secondPol & 0xff)] & 0xff);

    if (temp > 255) temp = temp - 255;

    return Exp[(temp & 0xff)];

 }

Now when i measure the performance speed of this method as below

 public void runABunch() {

    long start = System.nanoTime();
    int a=0x59;
   int b=0xb4;
     for (int i = 0; i < 5000000; i++)
        FFMulFast(a,b);
    long end = System.nanoTime();
    System.out.println("Time in nanoseconda: " + (end-start));
  }

The result of this one is approximately 0.1 seconds .

But when i run the same method runABunch but omitted for then i get totally different results and i do not understand why these results are totally different and which one is more reputable the one with for or the one without for

public void runABunch() {

long start = System.nanoTime();
int a=0x59;
  int b=0xb4;

    FFMulFast(a,b);
long end = System.nanoTime();
System.out.println("Time in nanoseconda: " + (end-start));
}

Also when i put some println statements the time of execution increased a lot.Anyone know why? I tried to use profiler like java visualvm but the problem is that i can not see anything there since it disappeared to me as soon as my application finished to run.

This may be caused by JIT optimizations. Are you doing a JIT warmup? — Argb32, May 26 '16 at 13:04
It depends on what the results are. Are you asking why the system time is different when you omit the `for` loop in `runABunch()`? — serebit, May 26 '16 at 13:05
@Deadshot yes that was what i am asking for. For example this method when i have for loop takes me 0.01 seconds but when i omit for in runABunch method then this method takes 0.0003 seconds. Why this difference? and which is the most reputable one out of this two? — Barlet South, May 26 '16 at 14:56
You're getting nothing related to the real performance. It's nothing but the measurement error. Any usable benchmark must take a few seconds at least, otherwise you measure just the warmup, which is completely irrelevant for a real program. But there are much more gotchas, so use JMH and read the linked post. — maaartinus, May 26 '16 at 15:06

score 0 · Answer 1 · answered May 26 '16 at 13:58

0

You shouldn't be trying to hand roll benchmark code like this. You should be using a framework like JMH instead.

answered May 26 '16 at 13:58

AlBlue

23,254
14
71
91

score 0 · Answer 2 · answered May 26 '16 at 15:03

The answer to your question is more simple than you might think. Your code with the for loop runs your FFMulFast() method five million times, while omitting the for loop runs the same method, but only once. Completing the same task five million times will always result in a longer elapsed time than if you only ran it once.

How to decide which execution time is more reputable while using System.nanoTime ()

2 Answers2