1

"C is fast, Java is slow", right?

I compared the times table example from Video2Brain's Objectiv-C 3.0 tutorial for a)  Eclipse/Java and b) XCode/Objectiv-C on a MacBook Pro Quadcore. The result absolutely surprised me. Java is 3 times faster than the Objective-C implementation. Java: 0.718s compared to Objective-C: 2.416s.

Question: What do I miss? How can this be explained? Thanks!

a) Java Code:

public static void main(String[] args) {

int timesTable[][] = new int[10][10];
long beginTime = System.currentTimeMillis();

// 10 000 000 x
for (int count = 0; count < 10000000; count++) {
   for (int row = 0; row < 10; row++) {
      for (int col = 0; col < 10; col++) {
         timesTable[row][col] = (row +1) * (col +1);
      }
   }
 }
 long endTime = System.currentTimeMillis();
 System.out.println("Time elapsed = " + (endTime - beginTime) + "ms");
 }

b) Objective-C Code

int main(int argc, char* argv[]) {
int timesTable[10][10];

CFAbsoluteTime beginTime = CFAbsoluteTimeGetCurrent();

 // 10 000 000 x
for (int count = 0; count < 10000000; count++) {
  for (int row = 0; row < 10; row++) {
    for (int col = 0; col < 10; col++) {
       timesTable[row][col] = (row +1) * (col +1);
    }
  }
}
 CFAbsoluteTime endTime = CFAbsoluteTimeGetCurrent();
 NSLog(@"Time elapsed = %f", endTime - beginTime);
}

UPDATE

@nhahtdh Compiler Options: Compiler: Apple LLVM compiler 4.2

@jlordo: I get same result, when I change to timesTable[row][column] = (row +1) * (column +1) * count

@uchuugaka: Right, I agree its basicly C compared to Java. I run the test five times both. System was restarted. No other applications running.

@Anoop Vaidya: Yes, newest version, I get the same times, no differ LLVM GCC 4.2 or Apple LLVM compiler 4.2

UPDATE2

@justin: Thanks, problem solved. Build Settings -> Optimization Level

Kaptain
  • 1,358
  • 12
  • 20
  • 6
    Just a guess, but Java probably detects that you are doing useless work, and bail out early. – nhahtdh Mar 07 '13 at 13:27
  • 1
    @nhahtdh yes, HotSpot JIT Compiler does a pretty well job on this. – jlordo Mar 07 '13 at 13:28
  • 1
    Another guess- Java is 3 times better? – vikingsteve Mar 07 '13 at 13:29
  • 1
    Another thing I forgot to mention: What options did you compile your Obj-C version with? – nhahtdh Mar 07 '13 at 13:30
  • 1
    What times do you get if you change the inner assignment to `timesTable[row][column] = (row +1) * (column +1) * count`; That would change memory content on every step, instead of one iteration and then many no-ops. – jlordo Mar 07 '13 at 13:33
  • Did you try obj-c code with GCC and LLVM newest version? – Anoop Vaidya Mar 07 '13 at 13:34
  • Slow at doing what? Something completely meaningless? – millimoose Mar 07 '13 at 13:37
  • What part is Objective-C? Core Foundation is a C framework. Using NSLog hardly counts as an Objective-C benchmark... You also should calculate the elapsed time outside of the programs. Let the system tell you execution time. Lastly, how many times did you run these, on what machines, with what other processes running? – uchuugaka Mar 07 '13 at 13:44
  • Good catch. Wall clock time benchmarking needs a bunch of measures to get rid of noise. (Several - 20? 100? - runs, spaced apart, discarding outliers.) – millimoose Mar 07 '13 at 13:47

2 Answers2

10

Your 'benchmark' is flawed. The C optimizer (for the ObjC test) should realize that the whole loop can be removed. Speed comparisons should be performed with optimizations enabled (e.g. -O3). When running the test with optimization enabled, the time elapsed for the above program is 0.0 seconds (yes, I tested and confirmed that).

A better benchmark must be devised.

justin
  • 104,054
  • 14
  • 179
  • 226
  • Not to mention that micro-benchmarks are generally a complete waste of time outside of very specific algorithm tests. Micro-benchmarks are entirely useless when it comes to measuring performance in any kind of a GUI based application. – bbum Mar 07 '13 at 16:22
5

Most likely it is an inaccurate benchmark.

Read this:

How do I write a correct micro-benchmark in Java?

Community
  • 1
  • 1
cowls
  • 24,013
  • 8
  • 48
  • 78