3

I am planning to micro benchmark my java code which involves several calls to local as well as remote database. I was about to use System.nanoTime() but started reading about the micro benchmarking frameworks such as jmh and caliper. Use of these frameworks is definitely recommended but from whatever (little) I read, it seems that we can benchmark only a complete method and also it allows us to do this non-invasively (w.r.t existing code) i.e., we need not litter existing code with the code/annotations of jmh/caliper.

I want to benchmark only specific pieces of code (statements) within some methods. Is it possible to do this with any of micro benchmarking frameworks? Please provide some insights into this.

Raghava
  • 947
  • 4
  • 15
  • 29
  • 1
    Might be more interested in a "profiler" than a benchmark tool. Benchmark cases are generally supposed to be treated as tools, like unit tests - not part of normal code. – user2864740 Jun 13 '14 at 01:02
  • @user2864740, Are there any good, free to use, command line based profilers available? – Raghava Jun 13 '14 at 03:02

2 Answers2

2

I guess, calls to a DB are usually expensive enough to eliminate most of the problem with microbenchmarking. So your approach was probably fine. If you're measuring it in production, repeating the measurement many times, and don't care about a few nanoseconds, stick with System.nanoTime.

You're doing something very different from microbenchmarking like e.g. I did here. You're not trying to optimize a tiny piece of code and you don't want to eliminate external influences.

Microbenchmarking a part of a method makes no sense to me, as a method gets optimized as a whole (and possibly also inlined). It's a different level.

I don't think any framework could help, all they can do in your case is automate the work, which you don't seem to need. Note that System.nanoTime may take several hundreds cycles (which is probably fine in your case).

Community
  • 1
  • 1
maaartinus
  • 44,714
  • 32
  • 161
  • 320
0

You can try using metrics from codehale. I found its easy to use and low overhead if you are using in certain configuration i.e. Exponentially decaying Reservoir.

Micro level and precise benchmarking does comes with an associated cost with it i.e. memory overhead at run time for sampling, benchmark might it self take time for calculation and and stats generation (ideal one would be offsetting that from stats) . But if you want to bench mark db connection which I don't think should be very frequent, metrics might be appropriate, I found its easy to use. and yes it is bit invasive but configurable.

Rohit Sachan
  • 1,178
  • 1
  • 8
  • 16
  • What advantages does this provide compared to using System.nanoTime()? I would only use it for measuring the time taken. – Raghava Jun 13 '14 at 03:03
  • 1
    I think when we microbench mark we might be in need of measuring one particular statement multiple times. i.e. how much time it took an avg in a day or or what was max-min time taken, 99%tile statistics. That too in a running server, that can be measured and analysed incase of using some framework. – Rohit Sachan Jun 13 '14 at 05:46