3

I have a jsperf test case, and the result are pretty confusing. I have three "snippets":

  • multiplication
  • division
  • control (neither operation is done)

and most of the time, they all come out about the same speed... even the control! I guessed that the JS JIT compiler was removing my "unnecessary" instructions when they didn't seem to have any effect; so I started accumulating the results, and logging them to the console when the test loop is done, e.g.

for (var i = 0; i < nNumbers; i++) {
  result += a[i] / b[i];
}
console.log(result);

But then, I got wildly differing results when the console was open from when it wasn't. The slowdown from the console logging seemed to overwhelm any other performance issues.

So I tried cranking up the number of iterations within each "snippet," to minimize the amount of logging relative to the operations I'm trying to test. But I still get no significant speed difference between the three snippets. Really, division and multiplication are both about the same speed as evaluating a constant?? I must be doing something wrong. Or jsperf is broken.

There are related questions already answered, but none that I've found specific to Javascript benchmarking.

Community
  • 1
  • 1
LarsH
  • 27,481
  • 8
  • 94
  • 152

1 Answers1

1

Don't put console.logs in your timed sections. It's horribly slow in comparisons to the operations you actually want to measure, so it skews your results. Also - as you noticed - it varies in timing when the console is open or not.

You can prevent deoptimisations by putting your results in a global array. The optimiser can only remove code that does not affect the outcome, which is impossible if it manipulates global state.

Of course, this still does not necessarily prevent loop-invariant code motion, so you also need to make sure that your timed code always operates on different data.

Bergi
  • 630,263
  • 148
  • 957
  • 1,375
  • Can you give an example of putting results in a global array? If I declare `var results;` in the `setup` code, is that a global variable, or do I have to put it in the Preparation code? Does it have to be an array as opposed to a single (scalar) variable? (As far as loop-invariant code is concerned, I believe my test snippets are already immune to this, as each iteration through the loop operates on `a[i]` and `b[i]`, where `i` is the loop variable.) – LarsH Mar 22 '16 at 18:22
  • Putting the declaration in the setup code might suffice, but is not truly global - depends on how advanced the optimiser is. A scalar that you add to should work as well, but a scalar that you only overwrite probably won't. – Bergi Mar 22 '16 at 18:31
  • I don't see a way to put results in a global variable. I tried moving the declaration of `result` (singular actually) out to the Preparation code (see http://jsperf.com/multiplication-vs-division-global-results) but now I get a `ReferenceError: result is not defined` in each snippet. – LarsH Mar 22 '16 at 19:05
  • Next I tried using `window.result` as the global variable for results: `window.result += ...` in each snippet. This should be a global variable. However I still get odd results... division is now faster than multiplication (and control comes in first or last ... different each time). – LarsH Mar 22 '16 at 19:09
  • Did you put the `var result` outside of the ` – Bergi Mar 22 '16 at 19:59
  • No, apparently that's what jsperf does with the Preparation code. Guess I should have added a script tag around it. – LarsH Mar 22 '16 at 20:06
  • 1
    Now that I've put `` into the Preparation code, and am accumulating answers in `result` instead of `window.result`, the results seem plausible in relation to each other... although they're 10x faster than they were before. I guess the problem is solved. – LarsH Mar 22 '16 at 20:17