3

So, I'm trying to port a C library (libnfc) to Java using SWIG.

I've got to the point of having a compiled shared library, and a basic "nfc_version()" method call will work. However, calling "nfc_init()" to set things up causes a SIGSEGV error. Calling the nfc library directly is fine.

The commands I used to generate the shared library:

swig -java -I../libnfc/include nfclib.i 
gcc -c -I/usr/lib/jvm/java-7-openjdk-i386/include/ -I/usr/lib/jvm/java-7-openjdk-i386/include/linux nfclib_wrap.c
gcc -shared nfclib_wrap.o ../build/libnfc/libnfc.so libnfc_wrap.so

The libnfc.i file:

%module nfc
%{
#include <nfc/nfc.h>
#include <nfc/nfc-types.h>
#include <nfc/nfc-emulation.h>
%}

%include <nfc/nfc.h>
%include <nfc/nfc-types.h>
%include <nfc/nfc-emulation.h>

I.e. it should include all the methods that libnfc provides.

Here is the error log I am getting: http://openetherpad.org/AyVDsO4XTg

Obviously, it may be that a specific solution may not be available from the information I have provided. But any suggestions of things to try would be really appreciated (I'm sort of at the end of my knowledge here).

Flexo
  • 87,323
  • 22
  • 191
  • 272
Sam
  • 3,453
  • 1
  • 33
  • 57
  • 1
    You say you're calling `nfc_init` - where do you get the `SWIGTYPE_p_nfc_context` it needs to call it? If you just write `new SWIGTYPE_p_nfc_context()` in Java it will make a *pointer* that points to nothing, which would probably explain your segfault. – Flexo Jan 20 '13 at 10:49
  • 1
    (The "natural" interface in Java would be possibly be for `nfc_init` to create a context and return that. The logical step from that would be for all functions which take an `nfc_context` as their first argument to be methods on the context that is returned with the argument passed in initially. SWIG can generate that code for you, if it's what you're trying to do. Either that or if you can only ever have one context per process then the Java library could handle that automatically for you, completely hiding the context parameter). I'm happy to elaborate on either. – Flexo Jan 20 '13 at 10:55
  • thank-you for your replies Flexo. You were right in your first comment! I was a little confused about the types that SWIG generated (i.e. - '_p' represents a pointer. The problem was that SWIG *hadn't* generated a nfc_context type, as the headers I had included in the '.i' file only included the typedef of the nfc_context type. Including another header with a full nfc_context definition in it got me passed this error and now I am having a link error (the new header I have included, nfc-internal.h is not in the 'include' directory of libnfc) – Sam Jan 20 '13 at 17:33
  • in your second point, you say SWIG can somehow hide the context parameter? is this challenging to do? would it be easier just to stick with the interface that libnfc defines? – Sam Jan 20 '13 at 17:35
  • 1
    WRT the link error you shouldn't need to show SWIG the real `nfc_context`. How do you normally allocate on in libnfc? Without any extra work that ought to "just work" once you've built the wrapper. – Flexo Jan 20 '13 at 17:37
  • nfc_context *context; nfc_init(&context); – Sam Jan 20 '13 at 17:59
  • that's the code in one of the example programs – Sam Jan 20 '13 at 17:59
  • just to return to your first comment here. looking at the example C given by the library, nfc_init *is* called with an uninitialized pointer. So why would handing an uninitialized SWIGTYPE_p_p_nfc_context give the segfault? – Sam Jan 21 '13 at 10:19
  • (by the way, I don't have that link error, I have got past the nfc_init command BUT I'm curious about what you said about not having to show SWIG the real nfc_context. If I understand you correctly, the definition in nfc-types.h (typedef struct nfc_context nfc_context;) should be enough to have SWIG generate the nfc_context type for java? – Sam Jan 21 '13 at 10:35
  • 1
    That's correct off you don't want to look inside the struct from Java ever. SWIG wouldn't be able to construct or delete one without help, but that's exactly what %extend does. – Flexo Jan 21 '13 at 17:40
  • I don't think i *do* want to look inside that struct from java, it's just passed around to other methods. Given the above usage from C, how is this not identical to the following in Java? (except that the java is ugly): nfc.nfc_init(new SWIGTYPE_p_p_nfc_context(SWIGTYPE_p_nfc_context.getCPtr(new SWIGTYPE_p_nfc_context()), true)); - is this some difference between meanings of 'uninitialized' between the two languages? – Sam Jan 21 '13 at 18:35
  • 1
    There's a gotcha in that code - if you have an anonymous object (`new SWIGTYPE_p_nfc_context()` here) GC can identify it as unused once the call to `getCPtr` completes and delete the object (releasing the memory it owned, which would be a pointer here) before the call to `nfc_init` has even happened. So your pointer to a pointer is potentially dangling already. Additionally since `getCPtr` always returns an `int` I think you might well have an accidental cast in there too - I'd try to avoid every calling `getCPtr` since it's just too easy to lose the type information. (I.e. cast to `intptr_t`) – Flexo Jan 21 '13 at 20:26
  • let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/23114/discussion-between-flexo-and-sam) – Flexo Jan 21 '13 at 20:57

2 Answers2

5

To always pass the same pointer in to a functions automatically it's fairly straightforward in SWIG. For example given the "header" file test.h, which captures the core part of your problem:

struct context; // only used for pointers

void init_context(struct context **ctx) { *ctx=malloc(1); printf("Init: %p\n", *ctx); }
void release_context(struct context *ctx) { printf("Delete: %p\n", ctx); free(ctx); }

void foo(struct context *ctx) { printf("foo: %p\n", ctx); }

We can wrap it and automatically cause a global context to be passed in everywhere one is expected by doing something like:

%module test

%{
#include "test.h"

// this code gets put in the generated C output from SWIG, but not wrapped:
static struct context *get_global_ctx() {
  static struct context *ctx = NULL;
  if (!ctx) 
    init_context(&ctx);
  return ctx;
}
%}

%typemap(in,numinputs=0) struct context *ctx "$1=get_global_ctx();"

%ignore init_context; // redundant since we call it automatically

%include "test.h"

This sets a typemap for struct context *ctx that instead of taking an input from Java automatically calls get_global_ctx() everywhere it matches.

That's probably sufficient to make a sane-ish interface for a Java developer to use, however it's less than ideal: it forces the context to be a global and means that no Java application can ever work with multiple contexts at once.

A nicer solution, given that Java is an OO language, is to make the context become a first class Object. We can also make SWIG generate such an interface for us although it's a little more convoluted. Our SWIG module file becomes:

%module test

%{
#include "test.h"
%}

// These get called automatically, no need to expose:
%ignore init_context;
%ignore delete_context;

// Fake struct to convince SWIG it should be an object:
struct context {
  %extend {
    context() {
      // Constructor that gets called when this object is created from Java:
      struct context *ret = NULL;
      init_context(&ret); 
      return ret;
    }
    ~context() {
      release_context($self);
    }
  }
};

%include "test.h"

and we can exercise this code successfully:

public class run {
  public static void main(String[] argv) {
    System.loadLibrary("test");
    context ctx = new context();
    // You can't count on the finalizer if it exits:
    ctx.delete();
    ctx = null;
    // System.gc() might also do the trick and in a longer
    // running app it would happen at some point probably.
  }
}

gives:

Init: 0xb66dab40
Delete: 0xb66dab40

In a dynamically typed language that would be the hard part done - we could use meta programming of one form or another to insert a the member functions as needed. Thus we would be able to say something like new context().foo(); entirely as expected. Java is statically typed though so we need something more. We can do this in SWIG in a number of ways:

  1. Accept that we can now call test.foo(new context()); quite happily - it looks a lot like C in Java still though so I'd suggest it might be a code smell if you end up writing lots of Java that looks like C.

  2. Use %extend to (manually) add the methods into the context class, the %extend in test.i becomes:

    %extend {
        context() {
          // Constructor that gets called when this object is created from Java:
          struct context *ret = NULL;
          init_context(&ret); 
          return ret;
        }
        ~context() {
          release_context($self);
        }
        void foo() {
          foo($self);
        }
      }
    
  3. As with %extend, but write the glue on the Java side, using a typemap:

    %typemap(javacode) struct context %{
      public void foo() {
        $module.foo(this);
      }
    %}
    

    (Note: this needs to be early enough in the interface file to work)

Notice that nowhere here have I shown SWIG the real definition of my context struct - it always defers to my "library" for anything where the real definition is required, thus the opaque pointer remains complete opaque.


A simpler solution to wrap the init_context with a double pointer would be to use %inline to provide an extra function that is only used in the wrapper:

%module test

%{
#include "test.h"
%}

%inline %{
  struct context* make_context() {
    struct context *ctx;
    init_context(&ctx);
    return ctx;
  }
%}

%ignore init_context;

%include "test.h"

Is sufficient to allow us to write the following Java:

public class run {
  public static void main(String[] argv) {
    System.loadLibrary("test");
    // This object behaves exactly like an opaque pointer in C:
    SWIGTYPE_p_context ctx = test.make_context();
    test.foo(ctx);
    // Important otherwise it will leak, exactly like C
    test.release_context(ctx);
  }
}

Alternative, but similar approaches would include using the cpointer.i library:

%module test

%{
#include "test.h"
%}

%include <cpointer.i>

%pointer_functions(struct context *,context_ptr);

%include "test.h"

Which you can then use as:

public class run {
  public static void main(String[] argv) {
    System.loadLibrary("test");
    SWIGTYPE_p_p_context ctx_ptr = test.new_context_ptr();
    test.init_context(ctx_ptr);
    SWIGTYPE_p_context ctx = test.context_ptr_value(ctx_ptr);
    // Don't leak the pointer to pointer, the thing it points at is untouched
    test.delete_context_ptr(ctx_ptr);
    test.foo(ctx);
    // Important otherwise it will leak, exactly like C
    test.release_context(ctx);
  }
}

There's also a pointer_class macro that is a little more OO than that and might be worth using instead. The point though is that you're providing the tools to work with the opaque pointer objects that SWIG uses to represent pointers it knows nothing about, but avoiding the getCPtr() calls which are inherently subverting the type system.

Flexo
  • 87,323
  • 22
  • 191
  • 272
  • wow. ok. thanks :) this could be a bit over my head. ill need to have a proper read through. nfc_init is only the first of many functions I'd like to use from libnfc, and I dont have a lot of time to be doing it 'properly'. so I may well need to go with a simpler, less OO approach. and wrap up the functionality in the actual java. – Sam Jan 20 '13 at 18:47
  • Ok, I've had a proper look and that's not as bad as I first thought. What I don't want to have to do is repeat this process for many objects and functions in this library, as I'm trying to build *up* from this library. Still, very useful and I will probably be implementing this at least for the most commonly used methods etc. If you don't mind, I'll wait for accepting this as the answer till I've got it working. Hopefully should have it done today :) – Sam Jan 21 '13 at 10:30
  • 1
    there are simpler ways to generate usable Java interfaces but they ugly Java in my view. I'll add that if you want though. I also have an idea for a neat feature that would simplify this massively even for static typing. – Flexo Jan 21 '13 at 17:43
  • @Sam - I added another two examples of ways you could wrap the same header in Java. My preference is always "make it feel like first class Java" for users, even if that makes writing the interface a little more complex up front though which is why I skipped over them initially. – Flexo Jan 21 '13 at 20:48
  • hi, ive accepted this as the answer as it really is a very nice answer and gives lots of examples for how to proceed to making a nice java interface. unfortunately I haven't been able to implement this as I've had to stop work in Java (it seems it is pretty impossible to pass callbacks to java functions through swig...) – Sam Jan 25 '13 at 16:58
  • @Sam I answered a question about callbacks not too long ago: http://stackoverflow.com/a/12251183/168175 – Flexo Jan 25 '13 at 18:39
  • haha, you've got the answer to everything. i should be able to adapt that to the C interface I've got with a little work I should think (I dont have a 'set' function as such, I have to put the callback inside a struct, so ill use your header example as part of the wrapper) – Sam Jan 26 '13 at 15:53
  • amazing, this deserve to have way more upvote. I guess fewer people run into this problem. – Helin Wang Feb 01 '16 at 22:06
0

So Flexo's answer is the proper way to solve this problem but SWIG also offers the "cpointer.i" module (described here: http://www.swig.org/Doc1.3/SWIGDocumentation.html#Library_nn3) which allowed me to hack together a quick solution so that I could test I had the basic library working. I thought i would put in this answer just for completeness and offer an alternative to anyone who stumbles across this question.

The need for this is because of what I would describe as an asymmetry in what is generated by SWIG. A base type object has a property, swigCPtr, which is the memory address of that object (a "self pointer"). Then to make a pointer, you just get swigCptr from the base type and pass it in the constructor for the pointer type (SWIGTYPE_p_X) that swig generated. BUT the pointer type ONLY exists in Java and just holds the swigCptr value. There is no 'c' block of memory holding that pointer. Hence, there is no equivalent of the swigCPtr in the pointer type. Ie there is no self pointer stored in the pointer type, in the same way that there IS a self pointer stored in the base type.

So you can't make a pointer to a pointer (SWIGTYPE_p_p_X) as you dont have an address of the pointer to pass it when constructing it.

My new '.i' file is as follows:

%module nfc
%{
#include <nfc/nfc.h>
#include <nfc/nfc-types.h>
#include <nfc/nfc-emulation.h>
%}

%include <nfc/nfc.h>
%include <nfc/nfc-types.h>
%include <nfc/nfc-emulation.h>

%include "cpointer.i"
%pointer_functions(nfc_context*, SWIGTYPE_p_p_nfc_context)

What this last macro does is provide 4/5 functions for making pointers. The functions all take and return the types that swig should already have generated. The new The usage in Java to get the nfc_init and nfc_open commands to work:

SWIGTYPE_p_p_nfc_context context_p_p = nfc.new_SWIGTYPE_p_p_nfc_context();
nfc.nfc_init(context_p_p);
//get the context pointer after init has set it up (the java doesn't represent what's happening in the c) 
SWIGTYPE_p_nfc_context context_p = nfc.SWIGTYPE_p_p_nfc_context_value(context_p_p);
SWIGTYPE_p_nfc_device pnd = nfc.nfc_open(context_p, null);

notice that I have to get the pointer from the double pointer after the init command has completed, as the information stored in the java pointer object is separated from the C 'world'. So retrieving the 'value' of context_p_p will give a context_p with the correct value of its pointer.

Sam
  • 3,453
  • 1
  • 33
  • 57