Create Java8 function reference programmatically

Question

Just a theoretic question, I do not have practical use-case currently.

Assuming some my API accepts function reference as an argument and I would like to both feed it directly from code via '::' syntax or collect matching functions via reflection, store in some Map and invoke conditionally.

It is possible to programmatically convert method into Consumer<String>?

Map<String, Consumer<String>> consumers = new HashMap<>();
consumers.put("println", System.out::println);

Method method = PrintStream.class.getMethod("println", String.class);
consumers.put("println", makeFunctionReference(method));
...
myapi.feedInto(consumers.get(someInput.getConsumerId()));

Update:

Though not satisfied by solutions in currently provided answers, but after getting the hint about LambdaMetaFactory I tried to compile this code

public class TestImpl {
    public static void FnForString(String arg) {}
}

public class Test {
    void test() {
        List<String> strings = new ArrayList<>();
        Consumer<String> stringConsumer = TestImpl::FnForString;

        strings.stream().forEach(stringConsumer);
        strings.stream().forEach(TestImpl::FnForString);
        stringConsumer.accept("test");
    }
}

and after feeding only Test class into CFR decompiler I'm getting following back:

public class Test {
    void test() {
        ArrayList strings = new ArrayList();
        Consumer<String> stringConsumer = 
            (Consumer<String>)LambdaMetafactory.metafactory(
                null, null, null, 
                (Ljava/lang/Object;)V, 
                FnForString(java.lang.String), 
                (Ljava/lang/String;)V)();
        strings.stream().forEach(stringConsumer);
        strings.stream().forEach(
            (Consumer<String>)LambdaMetafactory.metafactory(
                null, null, null, 
                (Ljava/lang/Object;)V, 
                FnForString(java.lang.String ), 
                (Ljava/lang/String;)V)());
        stringConsumer.accept("test");
    }
}

Out of that I see that:

This is somehow possible to do in '1-liner' manner
No exception handling is required
I have no idea what is (Ljava/lang/Object;)V (and others) in decompiler's output. It should match to MethodType in metafactory() arguments. Additionally - either decompiler 'eats/hides' something, but there seems to be now invocations of methods during getting of function reference.
(offtop) Obtaining function reference even in compiled code is at least one function call - in general this may be not unnoticeably cheap operation in performance critical code.

And ... with both Test and TestImpl classes provided, CFR reconstructs absolutely same code that I've compiled.

The first thing to understand is that method references are a compile time concept. In your first snippet, `println` is bound to the instance referenced by `out`. Is that also what you want with your second snippet? A `Method` has no relation to any instances. — Sotirios Delimanolis, Nov 20 '15 at 19:16
Good point, though I think it does not matter much in context of my question - I'm just curious if this possible even for static function references. — Xtra Coder, Nov 20 '15 at 19:37
what about this one : http://stackoverflow.com/q/21887358/4088809 — Tahar Bakir, Nov 20 '15 at 22:36
Try running the CFR-decompiled code. It would NPE -- if you could compile it at all. — Brian Goetz, Nov 21 '15 at 01:58
Yes - the partially decompiled invocation of `LambdaMetafactory.metafactory()` with 3 first arguments as `null` cause NPE. JavaDoc says for them - '_When used with {@code invokedynamic}, this is stacked automatically by the VM_.' So it is null in bytecode, not null at runtime. — Xtra Coder, Nov 21 '15 at 09:52

score 5 · Accepted Answer · edited May 23 '17 at 12:15

You could do this with reflection like this:

consumers.put("println", s -> {
    try {
        method.invoke(System.out, s);
    } catch (InvocationTargetException | IllegalAccessException e) {
        throw new RuntimeException(e);
    }
});

But it you want your code to compile to the same code using a method-reference (i.e. using invokedynamic instructions), you can use a MethodHandle. This does not have the overhead of reflection and so it will perform a lot better.

MethodHandles.Lookup lookup = MethodHandles.lookup();
MethodType methodType = MethodType.methodType(void.class, String.class);
MethodHandle handle = lookup.findVirtual(PrintStream.class, "println", methodType);

consumers.put("println", s -> {
    try {
        handle.invokeExact(System.out, s);
    } catch (Throwable e) {
        throw new RuntimeException(e);
    }
});

consumers.get("println").accept("foo");

In this code, first a MethodHandles.Lookup object is retrieved. This class is reponsible for creating MethodHandle objects. Then a MethodType object, which represents the arguments and return type accepted and returned by a method handle is created: in this case, it is a method that returns void (hence void.class) and takes a String (hence String.class). Finally, the handle is obtained by finding the println method on the PrintStream class.

You can refer to this question (and this one) for more information about what MethodHandles are.

You solution is actually the same as dealing with 'old java reflection' provided in another answer - having 'own' lambda to invoke 'handle.invokeExact()' is pretty much the same thing. It seems there is shorter way - please check the update question. — Xtra Coder, Nov 20 '15 at 22:37
@XtraCoder I'm not sure I understand. The exceptions are not going to disappear. Using a MethodHandle is definitely not the same thing than reflection. It is a completely different. In your update, note that `LambdaMetaFactory` deals with MethodHandles. — Tunaki, Nov 20 '15 at 23:02
Regarding exceptions - point is uncovered in another 'answer' - no exception handling is required in the real code with lambdas and function references is _because checked exceptions are a java "lie"_. In other respects your solution seems to be closes to what can be done in 3rd party code. — Xtra Coder, Nov 21 '15 at 16:12

score 3 · Answer 2 · edited May 23 '17 at 12:31

The simplest, albeit not necessarily most performant, approach would be just wrapping the Method into a Consumer.

final Method m = ...
final T target = ...

Consumer<String> c = (arg1) => m.invoke(t, arg1);

Using the LambdaMetaFactory may yield more optimal code, but considering that you're dispatching through a Map it's probably not worth it.

This is somehow possible to do in '1-liner' manner

If you really want to emulate what the bytecode does that's only true for for sufficiently tortured definitions of one-liner. Your decompiler lies to you to some extent.

No exception handling is required

That is because the concept of checked exceptions does not exist on the bytecode level. This can be emulated with static helper methods that do a sneaky rethrow for you.

I have no idea what is (Ljava/lang/Object;)V (and others) in decompiler's output. It should match to MethodType in metafactory() arguments. Additionally - either decompiler 'eats/hides' something, but there seems to be now invocations of methods during getting of function reference.

It looks like pseudocode for invokedynamic calls. What the JVM really does is more complicated and can't be expressed concisely in java since it involves lazy initialization. It's best to read the java.lang.invoke package description to get an idea what really happens.

The java-level equivalent to the linking stage would be putting the CalleSite's dynamicInvoker MH into a static final MethodHandle field and calling its invokeExact method.

(offtop) Obtaining function reference even in compiled code is at least one function call - in general this may be not unnoticeably cheap operation in performance critical code.

as mentioned above, the linking stage is equivalent to putting the methodhandle in a static field once and then calling that in the future instead of attempting to resolve the method a second time.

This is actually Java reflection wrapped into lambda and actually requires Exception handling during invocation, which is not supposed nor by 'println()' function nor by Consumer.accept(). Does not look to be a fine replacement. — Xtra Coder, Nov 20 '15 at 21:33
Method handles, LambdaMetaFactory and old school reflection methods all throw exceptions, so that makes little difference, you will have to handle them either way. I simply assume that you are sufficiently skilled to add them in the places where it is necessary. — the8472, Nov 20 '15 at 21:43
Yes, you are right - i know how to perform exception handling, and 'no' - raw compiled code somehow leaves absolutely without exception handling. Please check the update question. — Xtra Coder, Nov 20 '15 at 22:20
that's because checked exceptions are a java "lie", they don't exist on the bytecode level, so the lambdametafactory can just throw its exceptions without explicit handling. if you want to emulate that behavior too you can use a [sneaky throw](http://www.jayway.com/2010/01/29/sneaky-throw/). — the8472, Nov 20 '15 at 23:06
Thanks for comments, many good points, but ... i've tested both approaches recommended here (http://stackoverflow.com/questions/33844082) and wrapping reflection call into inline lambda has rather bad performance because of reflection. — Xtra Coder, Nov 21 '15 at 16:04
I said so from the start. But you mentioned you're dispatching through a `Map` instance and not through static fields, so that's not necessarily all that fast anyway. — the8472, Nov 21 '15 at 16:31

score 3 · Answer 3 · answered Nov 24 '15 at 18:10

The decompiler failed badly on your code, however, there is no correct decompiling anyway, besides recreating the original Java 8 method reference, which is not what you were interested in.

The lambda expressions and method references are compiled using an invokedynamic byte code instruction which has no equivalent in the Java programming language. The equivalent code would be something like:

public static void main(String... arg) {
    Consumer<String> consumer=getConsumer();
    consumer.accept("hello world");
}

static Consumer<String> getConsumer() {
    try {
        MethodHandles.Lookup lookup=MethodHandles.lookup();
        MethodType consumeString = MethodType.methodType(void.class, String.class);
        return (Consumer<String>)LambdaMetafactory.metafactory(lookup, "accept",
            MethodType.methodType(Consumer.class, PrintStream.class),
            consumeString.changeParameterType(0, Object.class),
            lookup.findVirtual(PrintStream.class, "println", consumeString), consumeString)
        .getTarget().invokeExact(System.out);
    }
    catch(RuntimeException | Error e) { throw e; }
    catch(Throwable t) { throw new BootstrapMethodError(t); }
}

except, that everything that is done within getConsumer() is originally handled by a single invokedynamic instruction which will treat all involved MethodHandles and MethodType instances like constants and whose result of the first-time evaluation gets an intrinsic caching facility. You can’t model that using ordinary Java source code.

Still, the Consumer<String> returned by the getConsumer() method above is the exact equivalent of the expression System.out::println (when assigned to a Consumer<String>) bearing the same behavior and performance characteristics.

You may study “Translation of Lambda Expressions” by Brian Goetz for getting a deeper understanding of how it works. Also, the API documentation of LambdaMetafactory is quite exhaustive.

Create Java8 function reference programmatically

3 Answers3