Why is my Scala tail-recursion faster than the while loop?

Question

Here are two solutions to exercise 4.9 in Cay Horstmann's Scala for the Impatient: "Write a function lteqgt(values: Array[Int], v: Int) that returns a triple containing the counts of values less than v, equal to v, and greater than v." One uses tail recursion, the other uses a while loop. I thought that both would compile to similar bytecode but the while loop is slower than the tail recursion by a factor of almost 2. This suggests to me that my while method is badly written.

import scala.annotation.tailrec
import scala.util.Random
object PerformanceTest {

  def main(args: Array[String]): Unit = {
    val bigArray:Array[Int] = fillArray(new Array[Int](100000000))
    println(time(lteqgt(bigArray, 25)))
    println(time(lteqgt2(bigArray, 25)))
  }

  def time[T](block : => T):T = {
    val start = System.nanoTime : Double
    val result = block
    val end = System.nanoTime : Double
    println("Time = " + (end - start) / 1000000.0 + " millis")
    result
  }

  @tailrec def fillArray(a:Array[Int], pos:Int=0):Array[Int] = {
    if (pos == a.length)
      a
    else {
      a(pos) = Random.nextInt(50)
      fillArray(a, pos+1)
    }
  }

  @tailrec def lteqgt(values: Array[Int], v:Int, lt:Int=0, eq:Int=0, gt:Int=0, pos:Int=0):(Int, Int, Int) = {
    if (pos == values.length)
      (lt, eq, gt)
    else
      lteqgt(values, v, lt + (if (values(pos) < v) 1 else 0), eq + (if (values(pos) == v) 1 else 0), gt + (if (values(pos) > v) 1 else 0), pos+1) 
  }

  def lteqgt2(values:Array[Int], v:Int):(Int, Int, Int) = {
    var lt = 0
    var eq = 0
    var gt = 0
    var pos = 0
    val limit = values.length
    while (pos < limit) {
      if (values(pos) > v)
        gt += 1
      else if (values(pos) < v)
        lt += 1
      else
        eq += 1
      pos += 1
    }
    (lt, eq, gt)
  }
}

Adjust the size of bigArray according to your heap size. Here is some sample output:

Time = 245.110899 millis
(50004367,2003090,47992543)
Time = 465.836894 millis
(50004367,2003090,47992543)

Why is the while method so much slower than the tailrec? Naively the tailrec version looks to be at a slight disadvantage, as it must always perform 3 "if" checks for every iteration, whereas the while version will often only perform 1 or 2 tests due to the else construct. (NB reversing the order I perform the two methods does not affect the outcome).

Answer 1

Test results (after reducing array size to 20000000)

Under Java 1.6.22 I get 151 and 122 ms for tail-recursion and while-loop respectively.

Under Java 1.7.0 I get 55 and 101 ms

So under Java 6 your while-loop is actually faster; both have improved in performance under Java 7, but the tail-recursive version has overtaken the loop.

Explanation

The performance difference is due to the fact that in your loop, you conditionally add 1 to the totals, while for recursion you always add either 1 or 0. So they are not equivalent. The equivalent while-loop to your recursive method is:

  def lteqgt2(values:Array[Int], v:Int):(Int, Int, Int) = {
    var lt = 0
    var eq = 0
    var gt = 0
    var pos = 0
    val limit = values.length
    while (pos < limit) {
      gt += (if (values(pos) > v) 1 else 0)
      lt += (if (values(pos) < v) 1 else 0)
      eq += (if (values(pos) == v) 1 else 0)
      pos += 1
    }
    (lt, eq, gt)
  }

and this gives exactly the same execution time as the recursive method (regardless of Java version).

Discussion

I'm not an expert on why the Java 7 VM (HotSpot) can optimize this better than your first version, but I'd guess it's because it's taking the same path through the code each time (rather than branching along the if / else if paths), so the bytecode can be inlined more efficiently.

But remember that this is not the case in Java 6. Why one while-loop outperforms the other is a question of JVM internals. Happily for the Scala programmer, the version produced from idiomatic tail-recursion is the faster one in the latest version of the JVM.

The difference could also be occurring at the processor level. See this question, which explains how code slows down if it contains unpredictable branching.

Answer 2

The two constructs are not identical. In particular, in the first case you don't need any jumps (on x86, you can use cmp and setle and add, instead of having to use cmp and jb and (if you don't jump) add. Not jumping is faster than jumping on pretty much every modern architecture.

So, if you have code that looks like

if (a < b) x += 1

where you may add or you may jump instead, vs.

x += (a < b)

(which only makes sense in C/C++ where 1 = true and 0 = false), the latter tends to be faster as it can be turned into more compact assembly code. In Scala/Java, you can't do this, but you can do

x += if (a < b) 1 else 0

which a smart JVM should recognize is the same as x += (a < b), which has a jump-free machine code translation, which is usually faster than jumping. An even smarter JVM would recognize that

if (a < b) x += 1

is the same yet again (because adding zero doesn't do anything).

C/C++ compilers routinely perform optimizations like this. Being unable to apply any of these optimizations was not a mark in the JIT compiler's favor; apparently it can as of 1.7, but only partially (i.e. it doesn't recognize that adding zero is the same as a conditional adding one, but it does at least convert x += if (a<b) 1 else 0 into fast machine code).

Now, none of this has anything to do with tail recursion or while loops per se. With tail recursion it's more natural to write the if (a < b) 1 else 0 form, but you can do either; and with while loops you can also do either. It just so happened that you picked one form for tail recursion and the other for the while loop, making it look like recursion vs. looping was the change instead of the two different ways to do the conditionals.