LambdaConversionException with generics: JVM bug?

Question

I have some code with a method reference that compiles fine and fails at runtime.

The exception is this:

Caused by: java.lang.invoke.LambdaConversionException: Invalid receiver type class redacted.BasicEntity; not a subtype of implementation type interface redacted.HasImagesEntity
    at java.lang.invoke.AbstractValidatingLambdaMetafactory.validateMetafactoryArgs(AbstractValidatingLambdaMetafactory.java:233)
    at java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:303)
    at java.lang.invoke.CallSite.makeSite(CallSite.java:289)

The class triggering the exception:

class ImageController<E extends BasicEntity & HasImagesEntity> {
    void doTheThing(E entity) {
        Set<String> filenames = entity.getImages().keySet().stream()
            .map(entity::filename)
            .collect(Collectors.toSet());
    }
}

The exception is thrown trying to resolve entity::filename. filename() is declared in HasImagesEntity. As far as I can tell, I get the exception because the erasure of E is BasicEntity and the JVM doesn't (can't?) consider other bounds on E.

When I rewrite the method reference as a trivial lambda, everything is fine. It seems really fishy to me that one construct works as expected and its semantic equivalent blows up.

Could this possibly be in the spec? I'm trying very hard to find a way for this not to be a problem in the compiler or runtime, and haven't come up with anything.

Answer 1

Here is a simplified example which reproduces the problem and uses only core Java classes:

public static void main(String[] argv) {
    System.out.println(dummy("foo"));
}
static <T extends Serializable&CharSequence> int dummy(T value) {
    return Optional.ofNullable(value).map(CharSequence::length).orElse(0);
}

Your assumption is correct, the JRE-specific implementation receives the target method as a MethodHandle which has no information about generic types. Therefore the only thing it sees is that the raw types mismatch.

Like with a lot of generic constructs, there is a type cast required on the byte code level which doesn’t appear in the source code. Since LambdaMetafactory explicitly requires a direct method handle, a method reference which encapsulates such a type cast cannot be passed as a MethodHandle to the factory.

There are two possible ways to deal with it.

First solution would be to change the LambdaMetafactory to trust the MethodHandle if the receiver type is an interface and insert the required type cast by itself in the generated lambda class instead of rejecting it. After all, it does similar for parameter and return types already.

Alternatively, the compiler would be in charge to create a synthetic helper method encapsulating the type cast and method call, just like if you had written a lambda expression. This is not a unique situation. If you use a method reference to a varargs method or an array creation like, e.g. String[]::new, they can’t be expressed as direct method handles and end up in synthetic helper methods.

In either case, we can consider the current behavior a bug. But obviously, compiler and JRE developers must agree on which way it should be handled before we can say on which side the bug resides.

Answer 2

I've just fixed this issue in JDK9 and JDK8u45. See this bug. The change will take a little while to percolate into promoted builds. Dan just pointed me at this Stack Overflow question, so I'm adding this note When you find bugs, please do submit them.

I addressed this by having the compiler create a bridge, as is the approach for many cases of complex method references. We are also examining spec implications.

Answer 3

This bug is not entirely fixed. I just ran into a LambdaConversionException in 1.8.0_72 and saw that there are open bug reports in Oracle's bug tracking system: link1, link2.

(Edit: The linked bugs are reported to be closed in JDK 9 b93)

As a simple workaround I avoid method handles. So instead of

.map(entity::filename)

I do

.map(entity -> entity.filename())

---

Here is the code for reproducing the problem on Debian 3.11.8-1 x86_64.

import java.awt.Component;
import java.util.Collection;
import java.util.Collections;

public class MethodHandleTest {
    public static void main(String... args) {
        new MethodHandleTest().run();
    }

    private void run() {
        ComponentWithSomeMethod myComp = new ComponentWithSomeMethod();
        new Caller<ComponentWithSomeMethod>().callSomeMethod(Collections.singletonList(myComp));
    }

    private interface HasSomeMethod {
        void someMethod();
    }

    static class ComponentWithSomeMethod extends Component implements HasSomeMethod {
        @Override
        public void someMethod() {
            System.out.println("Some method");
        }
    }

    class Caller<T extends Component & HasSomeMethod> {
        public void callSomeMethod(Collection<T> components) {
            components.forEach(HasSomeMethod::someMethod); //  <-- crashes
//          components.forEach(comp -> comp.someMethod());     <-- works fine

        }
    }
}

Answer 4

I found a workaround for this was swapping the order of the generics. For instance, use class A<T extends B & C> where you need to access a B method, or use class A<T extends C & B> if you need to access a C method. Of course, if you need access to methods from both classes, this won't work. I found this useful when one of the interfaces was a marker interface like Serializable.

As for fixing this in the JDK, the only info I could find were some bugs on openjdk's bug tracker that are marked resolved in version 9 which is rather unhelpful.

Answer 5

For the record, the Eclipse compiler of Eclipse 2021-09 (4.21.0) still seems to have this (or a very similar) bug, which I've reported here: https://bugs.eclipse.org/bugs/show_bug.cgi?id=577466

So, if you're developing with Eclipse, it might be this error still persists when developing (using the Eclipse compiler), even when at build time, it is absent (using javac via Maven or Gradle, etc.).