How does the memory management of closures in Scala work?

Question

Scala allows closure like

def newCounter = {
  var a=0
  () => {a+=1;a}
}

which defines a function that on every call returns a new independent counter function starting at 1:

scala> val counter1 = newCounter
counter1: () => Int = <function0>

scala> counter1()
res0: Int = 1

scala> counter1()
res1: Int = 2

scala> val counter2 = newCounter
counter2: () => Int = <function0>

scala> counter2()
res2: Int = 1

scala> counter1()
res3: Int = 3

This is quite impressive as usually a would be a representative of a memory address on the stack frame of newCounter. I've just read the closure chapter of "Programming in Scala" and it only has the following to say on that matter (p. 155):

The Scala compiler rearranges things in cases like this so that the captured parameter lives out on the heap, instead of the stack, and thus can outlive the method call that created it. This rearrangement is all taken care of automatically, so you don't have to worry about it.

Can anyone elaborate on how this works on byte code level? Is the access similar to a member variable of a class with all the associated synchronization and performance implications?

This is called the "funarg problem", and I'm guessing wiki might have some pointers about the theorethical background: https://en.wikipedia.org/wiki/Funarg_problem. The general solution seems to be "put the activation record or parts of it on the heap". (Google might also be able to find some lecture slides / notes or papers about this.) — millimoose, Jun 16 '13 at 17:40
Note SIP 21 "spores" (whatta name!) http://docs.scala-lang.org/sips/pending/spores.html — Gene T, Jun 16 '13 at 19:04
Related question: http://stackoverflow.com/questions/12831024/with-scala-closures-when-do-captured-variables-start-to-live-on-the-jvm-heap — Régis Jean-Gilles, Jun 17 '13 at 06:18

score 16 · Accepted Answer · edited May 23 '17 at 11:56

16

You could use scalac -Xprint:lambdalift <scala-file-name> to investigate this.

Your code is actually something like this:

def newCounter = {
  val a: runtime.IntRef = new runtime.IntRef(0);
  new Function0 {
    private[this] val a$1 = a
    def apply() = {
      a$1.elem = a$1.elem + 1
      a$1.elem
    }
  }
}

There is a wrapper for any var used by lambda. Other vars (not used in closures) are common locale variables.

The link to this wrapper is stored as field in the instance of function.

lambdalift in -Xprint:lambdalift is the compiler phase. You can get all phases with -Xshow-phases. You could use phase number instead of name, it's useful when you are not sure which phase you need.

edited May 23 '17 at 11:56

Community

1
1

answered Jun 16 '13 at 14:32

senia

37,745
4
88
129

What does the '20' mean in the cmd-line? – pedrofurla Jun 16 '13 at 17:24
Btw, it would be interest to contrast that with another outer variable that is not used in the closure. – pedrofurla Jun 16 '13 at 17:25
@pedrofurla: I've updated my answer. `20` was overkill, `lambdalift` (`15`) is enough. – senia Jun 16 '13 at 17:37
@pedrofurla: `What does the '20' mean in the cmd-line?`... It's a funny way to [suggest](http://stackoverflow.com/a/4023733/406435) [improvement](http://stackoverflow.com/a/4528092/406435). – senia Jun 16 '13 at 17:51
I wasn't suggesting an improvement, I was asking. I don't really care about SO rules much... – pedrofurla Jun 16 '13 at 20:00
@pedrofurla: I mean there is already an answer to your question in **your** answers. There are links to 2 your answers in my comment. – senia Jun 16 '13 at 20:05
@pedrofurla: I thought it's your way to suggest an improvement since you know an answer to the question. – senia Jun 16 '13 at 20:12
LOL. More or less, I've never seen numbers used there, that was my curiosity. – pedrofurla Jun 17 '13 at 04:06

How does the memory management of closures in Scala work?

1 Answers1

Linked