How is the distinction made which variables are passed by reference?

Question

After a long history with C and C++, I've only relatively recently become a programming-language polyglot started working in earnest with newer languages, specifically Groovy, and to a far lesser degree, Python.

RE: "pass-by-reference languages", I'd like to understand the rule (or rule-of-thumb) which variables are passed by function and which are passed by value ^.

I've written two equivalent programs in Jenkins/Groovy and Python. The programs demonstrate that an integer is passed by value, but a dictionary is passed by reference:

def dfunc(v) {
  v.foo = "dolor"
  v = [a:"a", b:"b"]
}

def podfunc(v) { v = 42 }

pipeline {
  agent any

  stages {
    stage( "1" ) {
      steps {
        script {
          def my_dict = [foo: "lorem", bar: "ipsum"]
          def my_int = 5

          echo my_dict.toString()
          dfunc(my_dict)
          echo my_dict.toString()

          echo my_int.toString()
          podfunc(my_int)
          echo my_int.toString()
        }
      }
    }
  }
}

def dfunc(d):
    d["foo"] = "dolor"
    d = { "a": "a", "b": "b" }
    
def podfunc(d):
    d = 42

my_dict = { "foo": "lorem", "bar": "ipsum" }
my_int = 5

print(my_dict)
dfunc(my_dict)
print(my_dict)

print(my_int)
podfunc(my_int)
print(my_int)

The output of both the above are effectively:

{'foo': 'lorem', 'bar': 'ipsum'}
{'foo': 'dolor', 'bar': 'ipsum'}
5
5

(with language-specific variation on bracing, etc.)

I think that another way of looking at this -- but I'm not sure if this is correct -- is perhaps that the languages aren't so much "pass-by-reference" as they are "pass-by-value, but certain variables are implicitly created on the heap".

In other words, I'm inferring that in Groovy and Python, when I call either dfunc(d) or podfunc(d), in both cases, I am technically passing by value; it's just "primitives" like the implicit integer my_int are created on the stack, and "other variables", like the implicit dictionary my_dict are created on the heap.

Is this a correct interpretation of why some variables appear to be passed by value and others passed by reference?

If so, which variables are implicitly created on the stack/heap, and why? And is this coder-controllable?
To my poor C-biased eyes, there's nothing about these two variables' declarations that self-evidences which is created on the stack and which is created on the heap:

def my_dict = [foo: "lorem", bar: "ipsum"]
def my_int = 5

The closest existing discussion I was able to find were these:
Groovy (or Java) - Pass by reference into a wrapper object
What's the difference between passing by reference vs. passing by value?
...but either I'm not fully appreciating the existing answers, or they're not quite addressing my question as stated above.

Thanks for helping me grok this.

---

^ @juanpa.arrivillaga corrected me: the programs do not demonstrate that anything is passed by reference - this question, inevitably, becomes about the misapplication of the phrase "pass by reference".

Answer 1

This answer focuses on Groovy/JVM, but I believe that conceptually it works similarly in Python and any other higher level language that does not expose pointers directly.

Groovy and Python (to my knowledge) do not have pointers a programmer can use within the language. However, variables of non-primitive types (at least in Groovy/Java), or Objects in other words, are passed "by reference" in a sense... except that you cannot access the pointer directly, so it's not possible to modify the pointer directly, you can only modify the data the Object itself points to by re-assigning them.

Objects are generally allocated on the heap, but JVM optimisations may actually skip allocation in what's called escape analysis - i.e. if it can prove the pointer is only needed in the local function, it may put the memory directly on the stack (TBH I am not sure when that's done and whether it's common). This shouldn't matter for the semantics of the program, though, because the difference between stack and heap allocation is not exposed to JVM programs.

Primitives, as you found out, are passed by value... immutable objects are also "semantically" passed by value (in fact, the JVM will add value types soon when Project Valhalla is ready), despite the fact that in reality they're just pointers: that's because you cannot reassign the pointer you're given, and the object being immutable means you cannot reassign any pointers it may have itself either: effectively, it's a value.

But when you pass a mutable object to a method, then you're allowed to change "pointers" inside the object (its fields), but not the data pointed to by the Object reference itself - because that's not exposed or accessible in Java/Groovy... so it's a bit similar to when you pass something by reference in languages that have pointers, but not the same.

A few examples:

import groovy.transform.Immutable 
import groovy.transform.Canonical

@Immutable
class Person {
    String name
}

@Canonical
class MutablePerson {
    String name
}

def takesPerson(Person p) {
    // nothing we can change on p as it's immutable,
    // effectively `p` is a "value", not reference.
    println p

    // we are allowed to re-assign p locally, but the
    // caller's reference is not affected. 
    p = null
}

def takesMPerson(MutablePerson p) {
    // we're allowed to change the internal structure of p
    p.name = 'Eva'

    // again, re-assigning p here won't change the caller's ref
    p = null
}

person = new Person(name: 'Joe')
mPerson = new MutablePerson(name: 'Mary')

println "Person before call: $person"
println "MutablePerson before call: $mPerson"

takesPerson person
takesMPerson mPerson

println "Person after call: $person"
println "MutablePerson after call: $mPerson"

Result:

Person before call: Person(Joe)
MutablePerson before call: MutablePerson(Mary)
Person(Joe)
Person after call: Person(Joe)
MutablePerson after call: MutablePerson(Eva)

---

To my poor C-biased eyes, there's nothing about these two variables' declarations that self-evidences which is created on the stack and which is created on the heap:

def my_dict = [foo: "lorem", bar: "ipsum"]
def my_int = 5

In Java, you know when something is probably going to go on the heap because you use the new keyword to create it (as opposed to literals which never need to go on the heap, and String which can be created with double-quotes in which case they MAY or MAY NOT allocate memory depending on whether it's a constant - static final without initialization logic - or it has been interned). But in Groovy, that's not the only case where data is likely being allocated on the heap because Groovy has more literals than Java, e.g. Lists, Sets and Maps, that also go on the heap (with caveats, as I mentioned earlier).

But if you just know that behind the scenes, Groovy collection literals are just calling Java's new CollectionType() and then adding the values you give into it, then you can have a reasonable guess at what is going to be allocated on the heap and what's not.

The more relevant question to the Groovy programmer is what data they can change and what's the effect of re-assigning something.

Hopefully, it's clear with the above examples what you can change: non-final local variables and class fields. And because method arguments are just local variables in JVM methods, re-assigning them has only effect in the scope of the method body. Re-assigning class fields has effect while the class instance is "alive" and until it's re-assigned again.

Data structures may seem to make this more complicated, but it's just the same thing as they are all implemented using classes and fields. When you change a Groovy Map (or dictionary), it's just re-assigning a field somewhere.

You can see that by implementing a little data structure, say an extremely simplified linked-list:

import groovy.transform.Canonical

@Canonical
class MyLinkedList {
  def value
  def next
}

def singleItemList = new MyLinkedList(value: 'single item')

def list = new MyLinkedList(value: 'first item',
                            next: singleItemList)

println singleItemList
println list

// modify list
list.next = new MyLinkedList(value: 'new item')

println list

// same kind of thing happens when you modify a Map
def map = [foo: 'lorem', bar: 'ipsum']

println map

map.foo = 'maximum'

println map

Result:

MyLinkedList(single item, null)
MyLinkedList(first item, MyLinkedList(single item, null))
MyLinkedList(first item, MyLinkedList(new item, null))
[foo:lorem, bar:ipsum]
[foo:maximum, bar:ipsum]