How can you extend Java to introduce passing by reference?

Question

Java is pass-by-value. How could you modify the language to introduce passing by reference (or some equivalent behavior)?

Take for example something like

public static void main(String[] args) {
    String variable = "'previous String reference'";
    passByReference(ref variable);
    System.out.println(variable); // I want this to print 'new String reference'
}

public static void passByReference(ref String someString) {
    someString = "'new String reference'";
}

which (without the ref) compiles to the following bytecode

  public static void main(java.lang.String[]);
    Code:
       0: ldc           #2                  // String 'previous String reference'
       2: astore_1
       3: aload_1
       4: invokestatic  #3                  // Method passByReference:(Ljava/lang/String;)V
       7: return

  public static void passByReference(java.lang.String);
    Code:
       0: ldc           #4                  // String 'new String reference'
       2: astore_0
       3: return

The code at 3: loads the reference onto the stack from the variable variable.

One possibility I'm considering is to have the compiler determine a method is pass by reference, possibly with ref, and change the method to accept a Holder object which stores the same reference as our variable. When the method completes, and possibly changes that reference in the holder, the variable on the caller side's value is replaced with the holder reference's value.

It should compile to an equivalent of this

public static void main(String[] args) {
    String variable = "'previous String reference'";
    Holder holder = Holder.referenceOf(variable);
    passByReference2(holder);
    variable = (String) holder.getReference(); // I don't think this cast is necessary in bytecode
    System.out.println(variable);
}

public static void passByReference(Holder someString) {
    someString.setReference("'new String reference'");
}

where Holder might be something like

public class Holder {
    Object reference;
    private Holder (Object reference) {
        this.reference = reference;
    }
    public Object getReference() {
        return this.reference;
    }
    public void setReference(Object reference) {
        this.reference = reference;
    }
    public static Holder referenceOf(Object reference) {
        return new Holder(reference);
    }
}

Where can this fail or how could you improve it?

Answer 1

The usual idiom I've seen for pass-by-reference in Java is to pass a single-element array, which will both preserve run-time type-safety (unlike generics which undergo erasure) and avoid the need to introduce a new class.

public static void main(String[] args) {
    String[] holder = new String[1];

    // variable optimized away as holder[0]
    holder[0] = "'previous String reference'";

    passByReference(holder);
    System.out.println(holder[0]);
}

public static void passByReference(String[] someString) {
    someString[0] = "'new String reference'";
}

Answer 2

To answer your question:

Where can this fail?

1. Final variables and enum constants
2. 'Special' references such as this
3. References that are returned from method calls, or constructed inline using new
4. Literals (Strings, integers, etc.)

...and possibly others. Basically, your ref keyword must only be usable if the parameter source is a non-final field or local variable. Any other source should generate a compilation error when used with ref.

An example of (1):

final String s = "final";
passByReference(ref s);  // Should not be possible

An example of (2):

passByReference(ref this);  // Definitely impossible

An example of (3):

passByReference(ref toString());  // Definitely impossible
passByReference(ref new String("foo"));  // Definitely impossible

An example of (4):

passByReference(ref "literal");  // Definitely impossible

And then there are assignment expressions, which seem to me like something of a judgement call:

String s;
passByReference(ref (s="initial"));  // Possible, but does it make sense?

It's also a little strange that your syntax requires the ref keyword for both the method definition and the method invocation. I think the method definition would be sufficient.

Answer 3

Your attempt to modify the language ignores the fact that this "feature" was explicitly left out to prevent well-known side-effect bugs from being able to happen in the first place. Java recommends to do what you are trying to archive by the use of data-holder classes:

public class Holder<T> {
  protected T value;

  public T getValue() {
    return value;
  }

  public void setValue(T value) {
    this.value = value;
  }
}

A thread-safe version would be the AtomicReference.

Now storing a single String in a class seems over-kill and most likely it is, however usually you have a data-holder class for several related values instead of a single String.

The big benefit of this approach is that what happens inside the method is very explicit. So even if you are programming on a Monday morning after an eventful weekend and the coffee machine just broke down, you still can tell easily what the code is doing (KISS), preventing several bugs from even happening in the first place, just because you forgot about that one feature of method foo.

If you think about what your approach can do that the data-holder version cannot, you'll soon realize that you are implementing something just because it is different, but effectively it has no real value.

Answer 4

Using AtomicReference class as holder object.

public static void main(String[] args) {
    String variable="old";
    AtomicReference<String> at=new AtomicReference<String>(variable);
    passByReference(at);
    variable=at.get();
    System.out.println(variable);
}

public static void passByReference(AtomicReference<String> at) {
  at.set("new");
}

Answer 5

Oddly enough, I've been thinking about this problem myself recently. I was considering whether it might be fun to create a dialect of VB that ran on the JVM - I decided it wouldn't be.

Anyway, there are two main cases where this is likely to be useful and well defined:

• local variables
• object attributes

I'm assuming that you're writing a new compiler (or adapting an existing one) for your new dialect of Java.

Local variables are typically handled by code similar to what you're proposing. I'm most familiar with Scala, which doesn't support pass-by-reference, but does support closures, which have the same issues. In Scala, there's a class scala.runtime.ObjectRef, which resembles your Holder class. There are also similar {...}Ref classes for primitives, volatile variables, and similar.

If the compiler needs to create a closure that updates a local variable, it "upgrades" the variable to a final ObjectRef (which can be passed to the closure in its constructor), and replaces uses of that variable by gets and updates by sets, on the ObjectRef. In your compiler, you could upgrade local variables whenever they're passed by reference.

You could use a similar trick with object attributes. Suppose that Holder implements an interface ByRef. When your compiler sees an object attribute being passed by reference, it could create an anonymous subclass of ByRef that reads and updates the object attribute in its get and set methods. Again, Scala does something similar to this for lazily evaluated parameters (like references, but read-only).

For extra brownie points, you could extend the techique to JavaBean properties and even Map, List and Array elements.

One side effect of this is that at the JVM level, your methods have unexpected signatures. If you compile a method with signature void doIt(ref String), at the bytecode level, you'll end up with the signature void doIt(ByRef) (you might expect this to be something like void doIt(ByRef<String>), but of course generics use type erasure). This can cause problems with method overloading, as all by-ref parameters compile to the same signature.

It may be possible to do this with bytecode manipulation, but there are pitfalls, like the fact that the JVM permits applications to re-use local variables - so at the bytecode level, it may not be clear whether a parameter is being re-assigned, or its slot re-used, if the application was compiled without debugging symbols. Also, the compiler may elide aload instructions if there's no possibility of a value having changed within the outer method - if you don't take steps to avoid this, changes to your reference variable may not be reflected in the outer method.

Answer 6

Think about how it might be implemented with a primitive type, say int. Java - the JVM, not just the language - does not have any "pointer" type to a local variable, on the frame (method stack) or the operand stack. Without that, it is not possible to truly pass by reference.

Other languages that support pass-by-reference use pointers (I believe, though I don't see any other possibility). C++ references (like int&) are pointers in disguise.

I've thought of creating a new set of classes that extend Number, containing int, long, etc. but not immutable. This could give some of the effect of passing primitives by reference - but they won't be auto-boxed, and some other features might not work.

Without support in the JVM, you can't have real pass-by-reference. Sorry, but that's my understanding.

BTW, there are already several Reference-type classes (like you'd like for Holder). ThreadLocal<> (which has get() and set()), or the Reference extenders, like WeakReference (which I think only have get()).

Edit: After reading some other answers, I'd suggest that ref be a form of auto-boxing. Thus:

class ReferenceHolder<T> {
    T referrent;
    static <T> ReferenceHolder<T> valueOf(T object) {
        return new ReferenceHolder<T>(object);
    }
    ReferenceHolder(T object) { referrent = object; }
    T get()            { return referrent; }
    void set(T value)  { referrent = value; }
}

class RefTest {
    static void main() {
        String s = "Hello";
        // This is how it is written...
        change(s);
        // but the compiler converts it to...
        ReferenceHolder<String> $tmp = ReferenceHolder.valueOf(s);
        change($tmp);
        s = $tmp.get();
    }
    // This is how it is written...
    static void change(ref Object s) {
        s = "Goodbye";              // won't work
        s = 17;             // *Potential ClassCastException, but not here*
    }
    // but the compiler converts it tothe compiler treats it as:
    static <T> void change(ReferenceHolder<T> obj) {
        obj.set((T) "Goodbye");     // this works
        obj.set((T) 17);    // *Compiler can't really catch this*
    }
}

But see where there is potential for putting the wrong kind of type in the ReferenceHolder? If genericized properly, the compiler may be able to warn sometimes, but as you likely want the new code to resemble normal code as much as possible, there is the possibility of a CCEx with each auto-ref call.

Answer 7

I think you can accomplish most of what you want by building an agent and using cglib.

Many of the examples given here can work. I'd recommend using the template you proposed because it will compile with the normal compiler.

public void doSomething(@Ref String var)

Then behind the scenes you use cglib to rewrite the annotated methods, which is easy. You'll also have to rewrite the caller, which i think will be much more complicated in cglib. javassist uses more of a "source code" oriented approach, and might be better suited for rewriting the callers.

Answer 8

Answering you question about how to extend the language my pick would be: - Using various holders technics as several other answers describe - Use annotations to attach metadata regarding which arguments should be passed by reference and then start juggling with a byte code manipulation library, like cglib in order to fulfil your ideas in byte code itself.

Though this whole idea seems strange.

Answer 9

There are several ways to write Java code as effectively pass-by-reference, even within the standard pass-by-value conventions.

One approach is to use instance or static variables whose scope includes a particular method, in lieu of explicit parameters. The variables which are being modified could be included in the comments, if you really want to mention their names at the beginning of a method.

The disadvantage with this approach is that the scope of these variables would need to encompass the entire class in question, rather than only the method. If you would like to restrict the variables' scopes more precisely, you could always modify them using getter and setter methods rather than as parameters.

Having worked with both Java and C/C++, I don't think Java's supposed inflexibility in being pass-by-value only is a big deal--for any programmers who know what happens to the variables, there are reasonable workarounds that can accomplish the same things functionally.

Answer 10

The question is about the language itself, but answers seems to mention practical tricks. Adding to the list.

There is an option to hold data in atomics, such as AtomicReference<T>

• new AtomicReference<>("my data")

Also it seems that Pair and other sorts of Tuples are good holders, so its often happens that you already have a holder in your project.

• Tuples.of(....) (reactor.util.function.Tuples)
• Pair.of(a, b) (spring, apache commons etc)
• AbstractMap.SimpleEntry (jdk)
• new Object[]{"my data"} can be a holder too

Answer 11

Java is (in fact) pass by reference. When the method is called, the reference(pointer) to the object is passed and when you modify the object you can see the modification when you return from the method. The problem with your example is that java.lang.String is immutable.

And what you are actually achieving with your example is output parameters.

Here is a slightly different version of Jeffrey Hantin:

public static void main(String[] args) {
  StringBuilder variable = new StringBuilder("'previous String reference'");
  passByReference(variable);
  System.out.println(variable); // I want this to print 'new String reference'
}

public static void passByReference(StringBuilder someString) {
  String nr = "'new String reference'";
  someString.replace(0, nr.length() - 1, nr);
}