JDK-18 Foreign Functions : How to upCall non-static functions?

Question

I've been playing around with the jdk.incubator.foreign stuff in JDK-18. It's pretty nice. WAY faster than JNI. An order of magnitude faster. The foreign memory stuff is better (and maybe slightly faster) than the UNSAFE stuff. Can't wait for this to ship.

One thing I can't figure out: How to upCall to a non-static JVM function?

If I can't, how do I pass some context down and back so I can cast the context into the correct instance/type in the java static function? In java how do you create a ValueLayout.ADDRESS from this? And vice-versa?

The only way I can figure out how to do it is to keep a list of instantiated classes, and round-trip the index in the list. Which seems like a hack.

Maybe the JVM reserves the right to move JVM memory around without restriction whenever it feels like it, so maybe it's not possible? If so, what is the recommended pattern for this?

--- edit - Add code example ---

public class RoundTripExample {

    private String name;
    RoundTripExample(String _name) {
        this.name = _name;
    }

    public void callback() {
        System.out.println("Called from native"+name);
    }

    public static void main(String[] args) throws Throwable {
        var fName = new File("../zig/zig-out/lib/libnativeFuncs.so").getCanonicalFile();
        System.load(fName.toString());

        var instance =new RoundTripExample("round trip name");

        CLinker link = CLinker.systemCLinker();

        SymbolLookup symLook = SymbolLookup.loaderLookup();

        ResourceScope scope = ResourceScope.newSharedScope();

        var nativeFunc = link.downcallHandle(
                symLook.lookup("zigCallTest").get(),
                FunctionDescriptor.of(ValueLayout.ADDRESS, ValueLayout.ADDRESS)
        );
        var handle =
                MethodHandles.lookup().findVirtual(
                        RoundTripExample.class,
                        "callback",
                        MethodType.methodType(void.class)
                );
        handle.bindTo(instance);
        var upcall = link.upcallStub(
                handle,
                FunctionDescriptor.ofVoid(ValueLayout.ADDRESS),
                ZigStubs.scope
        );

        nativeFunc.invoke(upcall);

    }
}

Assuming this is the way to do it, the problem is creating the upCall handle. You can only pass it a handle to functions that take LONG, INT, ADDRESS, etc. An instance of a class isn't any of those.

It would help if you could give an example of the method you're trying to call, and how you intend to call it. i.e. what is the general shape of the code in your use case? — Jorn Vernee, Apr 18 '22 at 23:00

Jorn Vernee · Accepted Answer · 2022-04-19T01:48:37.463

If the context you need to pass is constant for the target function, you could bind it to the target method handle using MethodHandles.insertArguments and then use the resulting method handle to create an upcall stub.

If the context is not constant for the target function, then your idea of using a list and passing around the index to the object you want is a pretty good way to go. This is essentially a way to turn an object into an integer, by inserting it into the list, and then back into an object by looking it up in the list again.

There's no safe way of turning an object into a plain native address, since, as you say, the object might be move by the JVM.

In the example, using bindTo looks feasible. In this case you've forgotten to assign the result of bindTo:

handle = handle.bindTo(instance);

Additionally, you'd have to drop the address argument passed from native code, as it looks like it's not being used by your Java code:

handle = MethodHandles.dropArguments(handle, 0, MemoryAddress.class);
var upcall = link.upcallStub(
    handle,
    FunctionDescriptor.ofVoid(ValueLayout.ADDRESS),
    ZigStubs.scope
);

Yep. That works. I guess `upcallStub` is creating a static function on the fly that is tracking the instance (`this`), so the native code doesn't need to pass any context though. Hmm... I'd prefer to pass the context though, but I guess there is no way to do that. — West_JR, Apr 19 '22 at 04:22
`upcallStub` dynamically generates some machine code which embeds a reference to the method handle you give it. That is essentially the static function created on the fly. The method handle target is invoked through a non-static method with the method handle as a receiver. The method handle instance can also contain data, so that is where the bound arguments are stored. After enough invocations though, the method handle will be customized, which means a dedicated class is created where the bound arguments are put in the constant pool. — Jorn Vernee, Apr 19 '22 at 09:25

JDK-18 Foreign Functions : How to upCall non-static functions?

1 Answers1