1

Following up from this thread: How should I write the .i file to wrap callbacks in Java or C#

I realize that my question is similar but the answer to that question was tailored specific to void* user data argument, while my callback takes an enum and a char*.

This is how my callback is defined and used in my header file

typedef void (*Callback_t)(const Log SomeLog, const char *Text);

virtual void SetCallback(const Callback_t SomeCallback, const Log SomeLog) = 0;

where Log is an enum.

I'm rather new to JNI and SWIG, so would need a specific guide on how to wrap this, similar to the one presented in the thread I mentioned above.

Thanks in advance.

Community
  • 1
  • 1
Armin Sh
  • 13
  • 4
  • it looks like your callback userdata argument is actually the `Log` typedef. Can you show us that too please? It probably has a void* inside, or can otherwise be massaged to give you a void*. Presumably since it's about logging the `char *` has to actually be a string and is the message that you need logging? – Flexo Oct 26 '16 at 15:10
  • `Log` is actually an enum, sorry my bad. `enum Log : uint32_t { blah = 1, blahblah = 2, blahblahblah = 3 };` And yes, `char *` is null terminated. – Armin Sh Oct 26 '16 at 15:21
  • right, but you don't control the contents of the `char *` string - that's how the application passes information to your callback? Can you change that? it's a pretty poorly designed interface if that's the case, but there are some workarounds possible – Flexo Oct 26 '16 at 15:23
  • Yes, correct. Also annotated by `_In_z_` if that makes it clearer. Unfortunately don't have access to the source file to change anything. – Armin Sh Oct 26 '16 at 15:27
  • One more question that's really puzzling - why is the `SetCallback` function virtual? I suspect it doesn't change anything here, but there are some weird design decisions here and the library authors really ought to be using `std::function` here instead to avoid a world of pain. Assuming SWIG's directors are out the picture there are two options open: Use a global variable (ugly) or use libffi (quite neat). I'll try and write both up later. – Flexo Oct 26 '16 at 15:35
  • I see another header which is inheriting stuff from this one and overriding some of the functions, probably that's why. Thanks in advance! – Armin Sh Oct 26 '16 at 15:45

1 Answers1

3

The simple solution is to adapt my answer at the previous question to just use a global variable to store the jobject we can't store inside some argument that gets passed to us during the callback. (It's bad design on the part of the library author that there doesn't seem to be a way of passing an argument in when setting the callback which is made available to the function, at the time a callback happens. Normally that's either void* or simply this. Given that it's C++ if it were me designing the library I'd have used std::function personally and then we could simply rely on SWIG directors here, but that doesn't seem to be an option in this scenario)

To work through this I wrote test.h:

typedef enum { 
  blah = 1,
  blahblah = 2,
} Log;

typedef void (*Callback_t)(const Log SomeLog, const char *Text);

void SetCallback(const Callback_t SomeCallback, const Log SomeLog);

void TestIt(const char *str);

and test.c:

#include "test.h"
#include <assert.h>

static Callback_t cb=0;
static Log log=0;

void SetCallback(const Callback_t SomeCallback, const Log SomeLog)  {
  cb = SomeCallback;
  log = SomeLog;
}

void TestIt(const char *str) {
  assert(cb);
  cb(log, str);
}

(Note that I'm working through this purely as an exercise in C since the C++ interface you've got to work with might as well be C to us here anyway).

With that in place you can then write an interface file for SWIG like:

%module test 

%{
#include "test.h"
#include <assert.h>

// NEW: global variables (bleurgh!)
static jobject obj;
static JavaVM *jvm;

// 2:
static void java_callback(Log l, const char *s) {
  printf("In java_callback: %s\n", s);
  JNIEnv *jenv = 0;
  // NEW: might as well call GetEnv properly...
  const int result = (*jvm)->GetEnv(jvm, (void**)&jenv, JNI_VERSION_1_6);
  assert(JNI_OK == result);
  const jclass cbintf = (*jenv)->FindClass(jenv, "Callback");
  assert(cbintf);
  const jmethodID cbmeth = (*jenv)->GetMethodID(jenv, cbintf, "Log", "(LLog;Ljava/lang/String;)V");
  assert(cbmeth);
  const jclass lgclass = (*jenv)->FindClass(jenv, "Log");
  assert(lgclass);
  const jmethodID lgmeth = (*jenv)->GetStaticMethodID(jenv, lgclass, "swigToEnum", "(I)LLog;");
  assert(lgmeth);
  jobject log = (*jenv)->CallStaticObjectMethod(jenv, lgclass, lgmeth, (jint)l);
  assert(log);
  (*jenv)->CallVoidMethod(jenv, obj, cbmeth, log, (*jenv)->NewStringUTF(jenv, s));
}

%}

// 3:
%typemap(jstype) Callback_t "Callback";
%typemap(jtype) Callback_t "Callback";
%typemap(jni) Callback_t "jobject";
%typemap(javain) Callback_t "$javainput";

// 4: (modified, not a multiarg typemap now)
%typemap(in) Callback_t {
  JCALL1(GetJavaVM, jenv, &jvm);
  obj = JCALL1(NewGlobalRef, jenv, $input);
  JCALL1(DeleteLocalRef, jenv, $input);
  $1 = java_callback;
}

%include "test.h"

In general that maps 1-1 onto the earlier answer, with the exception of replacing the struct holding callback information with global variables and improving the way we get JNIEnv inside the callback.

With our manually written Callback.java:

public interface Callback {
  public void Log(Log log, String str);
}

That's enough for this test case to compile and run successfully:

public class run implements Callback {
  public static void main(String[] argv) {
    System.loadLibrary("test");
    run r = new run();
    test.SetCallback(r, Log.blah);
    test.TestIt("Hello world");
  }

  public void Log(Log l, String s) {
    System.out.println("Hello from Java: " + s);
  }
}

Which works:

swig -Wall -java test.i
gcc -Wall -Wextra -o libtest.so -shared -I/usr/lib/jvm/java-1.7.0-openjdk-amd64/include/ -I/usr/lib/jvm/java-1.7.0-openjdk-amd64/include/linux/ test.c test_wrap.c  -fPIC 
javac *.java && LD_LIBRARY_PATH=. java run
In java_callback: Hello world
Hello from Java: Hello world

Since we don't like using global variables like this (multiple calls to SetCallback from the Java side will not behave the way we'd expect) my preferred solution (in a purely C world) is to use libffi to generate a closure for us. Essentially that lets us create a new function pointer for each active callback, so that the knowledge of which Java Object is being called can be implicit passed around each time a callback occurs. (This is what we're struggling to solve. Libffi has an example of closures that fits our scenario fairly closely.

To illustrate this the test case is unchanged, the SWIG interface file has become this:

%module test 

%{
#include "test.h"
#include <assert.h>
#include <ffi.h>

struct Callback {
  ffi_closure *closure;
  ffi_cif cif;
  ffi_type *args[2];
  JavaVM *jvm;
  void *bound_fn;
  jobject obj;
};

static void java_callback(ffi_cif *cif, void *ret, void *args[], struct Callback *cb) {
  printf("Starting arg parse\n");
  Log l = *(unsigned*)args[0];
  const char *s = *(const char**)args[1];
  assert(cb->obj);
  printf("In java_callback: %s\n", s);
  JNIEnv *jenv = 0;
  assert(cb);
  assert(cb->jvm);
  const int result = (*cb->jvm)->GetEnv(cb->jvm, (void**)&jenv, JNI_VERSION_1_6);
  assert(JNI_OK == result);
  const jclass cbintf = (*jenv)->FindClass(jenv, "Callback");
  assert(cbintf);
  const jmethodID cbmeth = (*jenv)->GetMethodID(jenv, cbintf, "Log", "(LLog;Ljava/lang/String;)V");
  assert(cbmeth);
  const jclass lgclass = (*jenv)->FindClass(jenv, "Log");
  assert(lgclass);
  const jmethodID lgmeth = (*jenv)->GetStaticMethodID(jenv, lgclass, "swigToEnum", "(I)LLog;");
  assert(lgmeth);
  jobject log = (*jenv)->CallStaticObjectMethod(jenv, lgclass, lgmeth, (jint)l);
  assert(log);
  (*jenv)->CallVoidMethod(jenv, cb->obj, cbmeth, log, (*jenv)->NewStringUTF(jenv, s));
}

%}

// 3:
%typemap(jstype) Callback_t "Callback";
%typemap(jtype) Callback_t "long";
%typemap(jni) Callback_t "jlong";
%typemap(javain) Callback_t "$javainput.prepare_fp($javainput)";

// 4:
%typemap(in) Callback_t {
  $1 = (Callback_t)$input;
}

%typemap(javaclassmodifiers) struct Callback "public abstract class"
%typemap(javacode) struct Callback %{
  public abstract void Log(Log l, String s);
%}

%typemap(in,numinputs=1) (jobject me, JavaVM *jvm) {
  $1 = JCALL1(NewWeakGlobalRef, jenv, $input);
  JCALL1(GetJavaVM, jenv, &$2);
}

struct Callback {
  %extend {
    jlong prepare_fp(jobject me, JavaVM *jvm) {
      if (!$self->bound_fn) {
        int ret;
        $self->args[0] = &ffi_type_uint;
        $self->args[1] = &ffi_type_pointer;
        $self->closure = ffi_closure_alloc(sizeof(ffi_closure), &$self->bound_fn);
        assert($self->closure);
        ret=ffi_prep_cif(&$self->cif, FFI_DEFAULT_ABI, 2, &ffi_type_void, $self->args);
        assert(ret == FFI_OK);
        ret=ffi_prep_closure_loc($self->closure, &$self->cif, java_callback, $self, $self->bound_fn);
        assert(ret == FFI_OK);
        $self->obj = me;
        $self->jvm = jvm;
      }
      return *((jlong*)&$self->bound_fn);
    }
    ~Callback() {
      if ($self->bound_fn) {
        ffi_closure_free($self->closure);
      }
      free($self);
    }
  }
};

%include "test.h"

Which has achieved our objective of removing globals by creating a closure using libffi. Callback has now become an abstract class, with a mix of C and Java components implementing it. The goal of it really is to hold the implementation of the abstract method Log and manage the lifecycle of the rest of the C data which needs to be held to implement this. Most of the libffi work is done inside a %extend SWIG directive, which pretty much mirrors the libffi documentation for closures. The java_callback function now uses the userdefined argument it gets passed in to store all of the information it needs instead of the global lookups, and has to pass/receive the function arguments via the ffi call. Our Callback_t typemap now makes use of the extra function we added via %extend to assist in setting up the function pointer to the closure that we really need.

One important thing to notice here is that you're responsible on the Java side for managing the lifecycle of the Callback instances, there is no way to make that information visible from the C side, so premature garbage collection is a risk.

To compile and run this the work implements needs to become extends in run.java and the compiler needs to have -lffi added. Other than that it works as before.


Since in your instance the language being wrapped is C++ not C we can actually simplify some of the JNI code a little by relying on SWIG's directors feature to assist us a little. This then becomes:

%module(directors="1") test

%{
#include "test.h"
#include <assert.h>
#include <ffi.h>
%}

%feature("director") Callback;

// This rename makes getting the C++ generation right slightly simpler
%rename(Log) Callback::call;

// Make it abstract
%javamethodmodifiers Callback::call "public abstract"
%typemap(javaout) void Callback::call ";"
%typemap(javaclassmodifiers) Callback "public abstract class"
%typemap(jstype) Callback_t "Callback";
%typemap(jtype) Callback_t "long";
%typemap(jni) Callback_t "jlong";
%typemap(javain) Callback_t "$javainput.prepare_fp()";
%typemap(in) Callback_t {
  $1 = (Callback_t)$input;
}

%inline %{
struct Callback {
  virtual void call(Log l, const char *s) = 0;
  virtual ~Callback() {
    if (bound_fn) ffi_closure_free(closure);
  }

  jlong prepare_fp() {
    if (!bound_fn) {
      int ret;
      args[0] = &ffi_type_uint;
      args[1] = &ffi_type_pointer;
      closure = static_cast<decltype(closure)>(ffi_closure_alloc(sizeof(ffi_closure), &bound_fn));
      assert(closure);
      ret=ffi_prep_cif(&cif, FFI_DEFAULT_ABI, 2, &ffi_type_void, args);
      assert(ret == FFI_OK);
      ret=ffi_prep_closure_loc(closure, &cif, java_callback, this, bound_fn);
      assert(ret == FFI_OK);
    }
    return *((jlong*)&bound_fn);
  }
private:
  ffi_closure *closure;
  ffi_cif cif;
  ffi_type *args[2];
  void *bound_fn;

  static void java_callback(ffi_cif *cif, void *ret, void *args[], void *userdata) {
    (void)cif;
    (void)ret;
    Callback *cb = static_cast<Callback*>(userdata);
    printf("Starting arg parse\n");
    Log l = (Log)*(unsigned*)args[0];
    const char *s = *(const char**)args[1];
    printf("In java_callback: %s\n", s);
    cb->call(l, s);
  }
};
%}

%include "test.h"

This .i file, which has vastly simplified the code required inside of java_callback is now a drop-in replacement for the previous libffi and C implementation. Pretty much all the changes are related to enabling directors sensibly and fixing a few C-isms. All we need to do now is call the pure virtual C++ method from within our callback and SWIG has generated code that handles the rest.

Community
  • 1
  • 1
Flexo
  • 87,323
  • 22
  • 191
  • 272
  • Thank you very much, really appreciate your effort. I'll use the last .i implementation. – Armin Sh Oct 27 '16 at 08:20
  • @arminsh FYI I've cleaned up the last .i implementation a fair bit since I originally wrote it – Flexo Oct 27 '16 at 20:16
  • using your latest implementation, I'm 99% percent of times getting `Unhandled exception at 0x00007FFF404AC3C3 (****.dll) in java.exe: 0xC0000005: Access violation reading location 0xFFFFFFFFFFFFFFFF.` I'm guessing it's about the premature garbage collection you warned me about? How should I manage the lifecycle of `run r = new run();` ? I've tried declaring `r` as `final` and `r.swigReleaseOwnership();` to no avail. Sometimes at random intervals the `Log` function works and I'm able to print the message. – Armin Sh Oct 30 '16 at 18:07
  • @ArminSh Does adding `System.out.println(r);` after the call to TestIt make it work reliably? If it does then that's definitely the garbage collection issue - and I'll have a think about a cleaner solution. – Flexo Oct 31 '16 at 08:11
  • I don't have the TestIT funtion but I put `System.out.println(r);` after `SetCallback()` and at the end of `main`, it didn't work. One thing that I guess need to be noted is that I added libffi through [nuget](https://www.nuget.org/packages/libffi/) to my VS2013 solution, since it was a pain to build libffi in Windows. – Armin Sh Oct 31 '16 at 09:37
  • found out it's a threading issue, apparently the callback is handled by a different thread on the C++ side. I managed to get this working using your global variable method by using `jvm->AttachCurrentThread()` before `jvm-GetEnv()` and then `jvm->DetachCurrentThread()` at the end. I rather use your libffi approach, would it be possible to change your libffi implementation to take care of this? – Armin Sh Oct 31 '16 at 13:03
  • Are you compiling with asserts turned off? That failure should be caught by my "pseudo error handling" via asserts. Anyway the GetEnv call should be fairly easy to do right instead of just assert inside `java_callback` of the libffi (but no directors variant). SWIG should generate the right code for the directors case: https://github.com/swig/swig/blob/master/Lib/java/director.swg#L147 – Flexo Oct 31 '16 at 18:04