Benchmark Asynchronous Code (Benchmark.js, Node.js)

Question

I'd like to use the Benchmark.js module to test some asynchronous code written in node.js. Specifically, I want to fire ~10,000 requests to two servers (one written in node, one written in PHP), and track how long it takes for each server to complete all requests.

I was planning to write a simple node script to fire these requests using Benchmark, but I'm a little confused regarding how to use it with asynchronous code. Usually in node modules, there's some sort of a callback that you call when your async code is complete, or a Promise is returned from the function etc. But with Benchmark, from everything I'm reading in the docs, it doesn't seem to handle async at all.

Does anyone know what I should be doing or looking at? I can write the benchmark manually if need be; it just seems like a common enough use case that Benchmark or others would probably have already implemented it in their professional-grade testing libraries.

Thanks for any direction, ~ Nate

score 16 · Accepted Answer · edited Jan 05 '16 at 15:53

16

It's not very well documented, but here's a PoC:

var Benchmark = require('benchmark');
var suite     = new Benchmark.Suite();

suite.add(new Benchmark('foo', {
  // a flag to indicate the benchmark is deferred
  defer : true,

  // benchmark test function
  fn : function(deferred) {
    setTimeout(function() {
      deferred.resolve();
    }, 200);
  }
})).on('complete', function() {
  console.log(this[0].stats);
}).run();

Benchmark.js v2 slightly changes the syntax:

var Benchmark = require('benchmark');
var suite = new Benchmark.Suite;

suite.add('foo', {
  defer: true,
  fn: function (deferred) {
    setTimeout(function() {
      deferred.resolve();
    }, 200);
  }
}).on('complete', function () {
  console.log(this[0].stats)
}).run()

edited Jan 05 '16 at 15:53

Pier-Luc Gendreau

13,553
4
58
69

answered Jul 25 '15 at 08:13

robertklep

198,204
35
394
381

1

Oh, okay - I did see the deferred object; didn't realize that's what it was for. Thanks! – opensourcejunkie Jul 27 '15 at 15:48
1

Yeah, like I said, it's not very well documented :-( FWIW, I just pushed the first version of [`benchr`](https://github.com/robertklep/node-benchr), which is a Mocha-like wrapper around Benchmark.js, to make things a bit more easy. It's still a WIP though. – robertklep Jul 27 '15 at 15:50
Oh awesome - I love mocha, and your syntax is definitely very familiar. I'll give it a go. What's the difference between benchr and bencha? – opensourcejunkie Jul 27 '15 at 17:46
@opensourcejunkie AFAIK, `bencha` doesn't support asynchronous test cases at all. I also don't like the way it requires you to create a `bench/` directory, and it's also not configurable. – robertklep Jul 27 '15 at 18:02
gotcha, thanks. I tried it out yesterday, and I too wasn't thrilled with the configuration options (e.g. how it always runs the index.js instead of offering something like mocha --grep). It does support async though, in the same style as mocha - a done callback specified in the test function parameters. Thanks for all your help, and I look forward to seeing your solution in the repo! – opensourcejunkie Jul 28 '15 at 16:53

score 2 · Answer 2 · answered Nov 26 '19 at 22:49

I came across this same issue when trying to test async functions. Here is an example of what I ended up using on Code Sand Box. Here is the link to the benchmark docs where it gives an example using the defer property.

And here is the code I used in Node.js for anyone who comes along and wants to see deffered used with async/await. I hope someone finds this useful!

const Benchmark = require('benchmark');
const suite = new Benchmark.Suite();
const promiseCount = 10;
const setupArray = Array.from(Array(promiseCount)).map((_, i) => i);

const sleep = (ms = 500) =>
  new Promise(resolve => {
    setTimeout(() => {
      resolve();
    }, ms);
  });

const asyncFunction = async (name, index) => {
  await sleep(100);
  return `${name}_${index}`;
};

suite
  .add("Promise.all", {
    defer: true,
    fn: async function(deferred) {
      const promiseArray = setupArray.map(asyncFunction);
      await Promise.all(promiseArray);
      deferred.resolve();
    }
  })
  .add("For loop", {
    defer: true,
    fn: async function(deferred) {
      const final = [];

      for (let i = 0; i < promiseCount; i++) {
        const data = await asyncFunction(setupArray[i], i);
        final.push(data);
      }
      deferred.resolve();
    }
  })
  .on("cycle", function(event) {
    console.log(String(event.target));
  })
  .on("complete", function() {
    console.log("Fastest is " + this.filter("fastest").map("name"));
  })
  .run({ async: true });

Benchmark Asynchronous Code (Benchmark.js, Node.js)

2 Answers2

Linked