11

I am trying to use Google Caliper to benchmark some simple code. I am using the examples from their websites. Here's what I've done so far:

  1. Downloaded the Caliper JAR and added it to my Netbeans project
  2. After having difficulties, I downloaded JUnit.jar and hamcrest.jar. Still not working.

Here's my code:

import com.google.caliper.Benchmark;

public class Benchmark1 extends Benchmark {

    public void timeNanoTime(int reps) {
      for (int i = 0; i < reps; i++) {
        System.nanoTime();
      }
    }

}

I am extending Benchmark because when I try to extend "SimpleBenchmark" like on their website it tells me it cannot find SimpleBenchmark. I then, in my main method, create a new Benchmark1() hoping something will happen. Nothing does. This is the code inside my main class.

Benchmark1 test = new Benchmark1();
test.timeNanoTime(10);

I know this is no doubt a simple error but I cannot, despite much Googling, figure out where I'm going wrong. The code compiles but does not run.

EDIT: I should say I'm running Netbeans on Windows 7 with Caliper 1.0

user2341412
  • 413
  • 2
  • 8
  • 14

2 Answers2

21

It's true; the documentation is woefully outdated and incomplete. I'm working on it. In the meantime, here's what will get your benchmark running.

Your main method should delegate to CaliperMain, not directly to the benchmark. Try

public static void main(String[] args) {
    CaliperMain.main(Benchmark1.class, args);
}

Windows will be a problem. Particularly, issue 215 will be the biggest blocker.

Scheintod
  • 7,953
  • 9
  • 42
  • 61
gk5885
  • 3,742
  • 21
  • 16
  • 9
    Windows is not supported? WTF Please post this as a big red notice on the project homepage. – Thorben May 08 '13 at 19:21
  • 2
    The current, pre-release version Caliper doesn't support Windows due to a bug. Support will return as we work through bugs, but none of the developers have Windows machines, so the turn-around has been slow. Moreover, Windows has been a lower priority as it is a less common production environment and somewhat poor benchmarking environment due to imprecision in the timers (similar issue with OS X). That said, since people develop on Windows it is still a goal to get it working. Please star the issue (and any others) as we consider those a factor in prioritizing work. – gk5885 May 13 '13 at 01:23
  • 1
    Nice to hear that the team is working on it. Didn't realize that caliper is in it's pre-release. It was a little bit frustrating to hear that windows does not work with caliper after spending time on building benchmarks and getting things to work. I share the opinion that Linux is smarter for production machines, but sometimes you code on your laptop. Most laptops come with Windows and special software from the manufacturerer relying on it. Thus it's more comfortable to use a ready to go System. Perhaps your developers can get a Windows from dreamspark or elsewhere. – Thorben May 15 '13 at 20:19
  • 3
    FWIW, the version @HEAD may work with Windows now if somebody wants to try it. – gk5885 May 23 '13 at 18:37
  • May I suggest that you add a notice to the homepage that the published jars don't support Windows, as suggested by Thorben? It's great that it's being worked on and/or fixed in head, but that won't help anyone who develops on Windows, sees a jar there and expects it to work. – Joe Kearney Feb 20 '14 at 11:17
  • Hey how about working with Annotations? How can i load a SNAPSHOT to use Annotations? – DenicioCode Feb 28 '14 at 11:25
  • I have been struggling a lot to find some documentation. Some tutorial on caliper as to how to start would be extremely helpful. – Sahil Arora Mar 15 '18 at 18:44
6

You could switch over to Perfidix http://perfidix.org/

Perfidix has eclipse Integration and can be used like JUnit.

Another option would be JUnitbenchmarks http://labs.carrotsearch.com/junit-benchmarks.html

It's a really great framework for Junit 4+. It can even build html charts to compare results.

simon04
  • 3,054
  • 29
  • 25
Thorben
  • 953
  • 13
  • 28