How do you deallocate memory in java (using jna) from a C String returned from cgo?

Question

Golang


import "C"

//export MyFunction
func MyFunction(str *C.char) *C.char {
    goStr := C.GoString(str)
    msg := goStr + " world"
    return C.CString(msg)
}

func main() {}

Java


public interface MyLib extends Library {

    MyLib INSTANCE = Native.load("mylib.so", MyLib.class);

    String MyFunction(String str);

}

public class CGoExample {

    public static void main(String[] args) {
        String str = "Hello";
        String msg = MyLib.INSTANCE.MyFunction(str);
        System.out.println(msg);
    }

}

This code works, the Java String is passed to Go, and Go returns a new String. The issue I'm having is that the CString returned by Go will not be deallocated by either the Java or Go garbage collector. The CGO documentation states "it is the caller's responsibility to arrange for it to be freed, such as by calling C.free", of course in my case calling C.free will not work as I'm returning the CString and if I free the CString before returning it I don't get a proper response from the Go function. I also tried calling defer C.free(unsafe.Pointer(msg)) but as I found out the hard way defer statements will be called before the return statement.

As I need to return the CString I'm assuming its memory needs to be deallocated from java, how would I go about doing this?

Edit:

Based on DanielWiddis' comment I have tried the following but the java process exists with code -1073741819.

// #include <stdlib.h>
import "C"

import "unsafe"

//export MyFunction
func MyFunction(str *C.char) *C.char {
    goStr := C.GoString(str)
    msg := goStr + " world"
    return C.CString(msg)
}

//export Free
func Free(str *C.char){
    C.free(unsafe.Pointer(str))
}

func main() {}

public interface MyLib extends Library {

    MyLib INSTANCE = Native.load("mylib.so", MyLib.class);

    String MyFunction(String str);

    void Free(String str);

}

public class CGoExample {

    public static void main(String[] args) {
        String str = "Hello";
        String msg = MyLib.INSTANCE.MyFunction(str);
        System.out.println(msg);
        MyLib.INSTANCE.Free(msg);
    }

}

In older versions of Java I would probably use a `finalize()` method for this. However I think `finalize()` is deprecated now, so you'll need to do a bit of extra work. Or at least prepare for the day that `finalize()` is removed entirely, if you don't want to do the work yet. — markspace, Feb 20 '21 at 17:19
Yeah here we go: https://stackoverflow.com/questions/56139760/why-is-the-finalize-method-deprecated-in-java-9 — markspace, Feb 20 '21 at 17:37
Can you not just map `C.free` as another function/method in the interface and call it from Java? That's how just about every other API does it. — Daniel Widdis, Feb 20 '21 at 19:00
In theory, `CString` does really go for memory to the `malloc()` provided by the `libc` which `cgo` arranges for to be linked with the resulting executable image; `C.free` is an (almost) 1-to-1 map to `free()` of the same `libc`. Hence _technically_ you can just call `free()` on that memory but this has to be _the same_ `free()`—in the sense that it must be the same symbol provided by the same `libc`. I have no idea whether you can guarantee this with JNA. If not, I'm with the Daniel Widdis' suggestion. — kostix, Feb 20 '21 at 19:08
One another alternative is to notice that `C.CString(s)` is nothing special (call `malloc()` to obtain a chunk of memory of the length `len(s)+1`, copy over the `s`'s bytes and write NUL at the end) and provide to the C side your own "allocating" and "deallocating" functions which would work just like `malloc()` and `free()` and will be shared by both the JNA part and the Go part. The Go part can then have a thin wrapper around that allocator function which would do what `C.CString` does: call the allocator, copy the memory, write NUL. — kostix, Feb 20 '21 at 19:13
Well, and if you control the JNA part, you could instead try to directly accomodate Go's strings which are just `struct`s with two fields: a pointer to the memory containing the string bytes and a platform-size integer containing the number of bytes. They are passed by value. — kostix, Feb 20 '21 at 19:16
@DanielWiddis I have edited my question and have provided code for what I think you suggested as a solution, however, I seem to be having some issues would you be able to provide a code example or correct any obvious mistakes you can see. — D0M1N0., Feb 20 '21 at 19:52
You won't be able to pass a java `String` as an argument like you do. It carries no "pointer" information about the native String. I think you may be able to do it if you change the `msg` variable to a `byte[]` array, but the best thing to do is probably just return the plain `Pointer` to the C string, read from it with `Pointer.getString(0)` and pass the same pointer to free it. — Daniel Widdis, Feb 20 '21 at 21:45

Daniel Widdis · Answer 1 · 2021-02-21T02:26:42.580

The return type of C.CString() is a pointer: *C.char.

By mapping this directly to a String on the Java side, you are losing track of that native Pointer, making it impossible to track and free it later.

The most straightforward solution is just to map that to a Pointer on the Java side. In the interface:

Pointer MyFunction(String str);

void Free(Pointer str);

Then to use it:

Pointer pMsg = MyLib.INSTANCE.MyFunction(str);
System.out.println(pMsg.getString(0));
MyLib.INSTANCE.Free(pMsg);

It's possible a byte[] array of (8 bit) characters would also work in this situation. Personally, I'd probably use a type safe pointer with some object oriented helper methods such as:

class CStringPtr extends PointerType {
    public CStringPtr(String str) {
        super(MyLib.INSTANCE.MyFunction(str));
    }

    public String getString() {
        return this.getPointer().getString(0);
    }

    public void free() {
        MyLib.INSTANCE.Free(this.getPointer());
    }
}

which would make your code:

CStringPtr pMsg = new CStringPtr(str);
System.out.println(pMsg.getString());
pMsg.free();

Thanks for your help! I'm going to mark mine as the correct answer just because its a full code solution which I feel most people will find more useful, although you gave a great answer which really helped me so I will upvote! — D0M1N0., Feb 20 '21 at 22:48
@D0M1N0. That's totally fine to accept your own more complete answer, but see my comments on it to improve it. I think you're being a bit aggressive with the memory freeing. And I think there's some value to the object oriented approach in hiding the details from the user. — Daniel Widdis, Feb 20 '21 at 23:11

D0M1N0. · Accepted Answer · 2021-02-23T19:59:53.453

1

Based on DanielWiddis' comments I came up with the following solution which successfully deallocates memory for all involved objects.

package main

// #include <stdlib.h>
import "C"

import (
    "unsafe"
)

//export MyFunction
func MyFunction(cStr *C.char) *C.char {
    str := C.GoString(cStr)
    msg := str + " world"
    return C.CString(msg)
}

//export Free
func Free(str *C.char){
    C.free(unsafe.Pointer(str))
}

func main() {}

public interface MyLib extends Library {

    MyLib INSTANCE = Native.load("mylib.so", MyLib.class);

    Pointer MyFunction(Pointer ptr);

    void Free(Pointer ptr);

}

public class CGoExample {

    public static void main(String[] args) {

        // Create pointer with a String value
        String str = "Hello";
        Pointer ptr = new Memory(str.length() + 1);
        ptr.clear();
        ptr.setString(0, str);

        //Pass the pointer to Go function which returns the pointer of the message
        Pointer resultPtr = MyLib.INSTANCE.MyFunction(ptr);

        //Get the message String from the pointer
        String msg = resultPtr.getString(0);

        //Deallocate the memory for the message
        MyLib.INSTANCE.Free(resultPtr);

        System.out.println(msg);
    }

}

edited Feb 23 '21 at 19:59

answered Feb 20 '21 at 22:42

D0M1N0.

51
5

Not a good idea to use `Native.free()` when you're allocating the memory on the Java side using the `Memory` class (which releases the memory when that Java object is GC'd). I suspect if you run your program long enough to trigger GC on that `Memory` objecct you'll get invalid memory access or segfault, etc. from the double-free. Also it looks like you're freeing that native memory before reading it with `getString()` which is also dangerous. – Daniel Widdis Feb 20 '21 at 23:07
Thanks for the comment! I removed `Native.free()`, but I'm already reading the string value from `resultPtr` before freeing it so haven't changed anything regarding that. – D0M1N0. Feb 21 '21 at 11:35
Almost there! One last change you need: instantiating `new Memory()` does not clear out that memory, so you are not guaranteed the last byte is null, and your `MyFunction` could lead to reading unallocated memory and undefined results. You should either `ptr.clear();` or `ptr.setByte(str.length(), (byte) 0);` before setting the string. – Daniel Widdis Feb 22 '21 at 17:56
Thanks again, I've edited the answer adding `ptr.clear()`, and have now marked correct! – D0M1N0. Feb 23 '21 at 20:03

How do you deallocate memory in java (using jna) from a C String returned from cgo?

2 Answers2