Why can method reference use non-final variables?

Question

I had some confusion about inner classes and lambda expression, and I tried to ask a question about that, but then another doubt arose, and It's probable better posting another question than commenting the previous one.

Straight to the point: I know (thank you Jon) that something like this won't compile

public class Main {
    public static void main(String[] args) {
        One one = new One();

        F f = new F(){      //1
            public void foo(){one.bar();}   //compilation error
        };

        one = new One();
    }
}

class One { void bar() {} }
interface F { void foo(); }

due to how Java manages closures, because one is not [effectively] final and so on.

But then, how come is this allowed?

public class Main {
    public static void main(String[] args) {
        One one = new One();

        F f = one::bar; //2

        one = new One();
    }
}

class One { void bar() {} }
interface F { void foo(); }

Is not //2 equivalent to //1? Am I not, in the second case, facing the risks of "working with an out-of-date variable"?

I mean, in the latter case, after one = new One(); is executed f still have an out of date copy of one (i.e. references the old object). Isn't this the kind of ambiguity we're trying to avoid?

Answer 1

A method reference is not a lambda expression, although they can be used in the same way. I think that is what is causing the confusion. Below is a simplification of how Java works, it is not how it really works, but it is close enough.

Say we have a lambda expression:

Runnable f = () -> one.bar();

This is the equivalent of an anonymous class that implements Runnable:

Runnable f = new Runnable() {
    public void run() {
       one.bar();
    }
}

Here the same rules apply as for an anonymous class (or method local class). This means that one needs to effectively final for it to work.

On the other hand the method handle:

Runnable f = one::bar;

Is more like:

Runnable f = new MethodHandle(one, one.getClass().getMethod("bar"));

With MethodHandle being:

public class MethodHandle implements Runnable {
    private final Object object;
    private final Method method;

    public MethodHandle(Object object, java.lang.reflect.Method method) {
        this.object = Object;
        this.method = method;
    }

    @Override
    public void run() {
        method.invoke(object);
    }
}

In this case, the object assigned to one is assigned as part of the method handle created, so one itself doesn't need to be effectively final for this to work.

Answer 2

Your second example is simply not a lambda expression. It's a method reference. In this particular case, it chooses a method from a particular object, which is currently referenced by the variable one. But the reference is to the object, not to the variable one.

This is the same as the classical Java case:

One one = new One();
One two = one;
one = new One();

two.bar();

So what if one changed? two references the object that one used to be, and can access its method.

Your first example, on the other hand, is an anonymous class, which is a classical Java structure that can refer to local variables around it. The code refers to the actual variable one, not the object to which it refers. This is restricted for the reasons that Jon mentioned in the answer you referred to. Note that the change in Java 8 is merely that the variable has to be effectively final. That is, it still can't be changed after initialization. The compiler simply became sophisticated enough to determine which cases will not be confusing even when the final modifier is not explicitly used.

Answer 3

The consensus appears to be that this is because when you do it using an anonymous class, one refers to a variable, whereas when you do it using a method reference, the value of one is captured when the method handle is created. In fact, I think that in both cases one is a value rather than a variable. Let's consider anonymous classes, lambda expressions and method references in a bit more detail.

Anonymous classes

Consider the following example:

static Supplier<String> getStringSupplier() {
    final Object o = new Object();
    return new Supplier<String>() {
        @Override
        public String get() {
            return o.toString();
        }
    };
}

public static void main(String[] args) {
    Supplier<String> supplier = getStringSupplier();
    System.out.println(supplier.get());  // Use o after the getStringSupplier method returned.
}

In this example, we are calling toString on o after the method getStringSupplier has returned, so when it appears in the get method, o cannot refer to a local variable of the getStringSupplier method. In fact it is essentially equivalent to this:

static Supplier<String> getStringSupplier() {
    final Object o = new Object();
    return new StringSupplier(o);
}

private static class StringSupplier implements Supplier<String> {
    private final Object o;

    StringSupplier(Object o) {
        this.o = o;
    }

    @Override
    public String get() {
        return o.toString();
    }
} 

Anonymous classes make it look as if you are using local variables, when in fact the values of these variables are captured.

In contrast to this, if a method of an anonymous class references the fields of the enclosing instance, the values of these fields are not captured, and the instance of the anonymous class does not hold references to them; instead the anonymous class holds a reference to the enclosing instance and can access its fields (either directly or via synthetic accessors, depending on the visibility). One advantage is that an extra reference to just one object, rather than several, is required.

Lambda expressions

Lambda expressions also close over values, not variables. The reason given by Brian Goetz here is that

idioms like this:

int sum = 0;
list.forEach(e -> { sum += e.size(); }); // ERROR

are fundamentally serial; it is quite difficult to write lambda bodies like this that do not have race conditions. Unless we are willing to enforce -- preferably at compile time -- that such a function cannot escape its capturing thread, this feature may well cause more trouble than it solves.

Method references

The fact that method references capture the value of the variable when the method handle is created is easy to check.

For example, the following code prints "a" twice:

String s = "a";
Supplier<String> supplier = s::toString;
System.out.println(supplier.get());
s = "b";
System.out.println(supplier.get());

Summary

So in summary, lambda expressions and method references close over values, not variables. Anonymous classes also close over values in the case of local variables. In the case of fields, the situation is more complicated, but the behaviour is essentially the same as capturing the values because the fields must be effectively final.

In view of this, the question is, why do the rules that apply to anonymous classes and lambda expressions not apply to method references, i.e. why are you allowed to write o::toString when o is not effectively final? I do not know the answer to that, but it does seem to me to be an inconsistency. I guess it's because you can't do as much harm with a method reference; examples like the one quoted above for lambda expressions do not apply.

Answer 4

No. In your first example you define the implementation of F inline and try to access the instance variable one.

In the second example you basically define your lambda expression to be the call of bar() on the object one.

Now this might be a bit confusing. The benefit of this notation is that you can define a method (most of the time it is a static method or in a static context) once and then reference the same method from various lambda expressions:

msg -> System.out::println(msg);