java.lang.UnsatisfiedLinkError even though libraries and classes have method

Question

I am using Python and Py4J to test JNI code. But when I call the JNI code I get the following error:

py4j.protocol.Py4JJavaError: An error occurred while calling o37.createInstance.
: java.lang.UnsatisfiedLinkError: com.mgr_api_JNI.createInstance(Lcom/mgr_api_types$EDisplayType;Ljava/lang/String;Lcom/mgr_api_types$ECommType;Ljava/lang/String;)V
    at com.mgr_api_JNI.createInstance(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
    at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
    at py4j.Gateway.invoke(Gateway.java:282)
    at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
    at py4j.commands.CallCommand.execute(CallCommand.java:79)
    at py4j.GatewayConnection.run(GatewayConnection.java:238)
    at java.base/java.lang.Thread.run(Thread.java:834)

I have looked at these links link 1, link 2, link 3, link 4, link 5, and link 6, plus others, but none of them solve my problem.

Code

mgr_api_JNI.java:

package com;
import com.mgr_api_types.*;

public class mgr_api_JNI
{
    static
    {
        try
        {
            System.loadLibrary("Mngr"); // Use "-Djava.library.path=[path to library]" option to load this library
        }
        catch (UnsatisfiedLinkError e)
        {
            System.err.println("Native code library 'Mngr' failed to load.\n" + e);
        }
    }

    public native void createInstance(com.mgr_api_types.EDisplayType displayType,
                                      String displaySerialNumber,
                                      com.mgr_api_types.ECommType commType,
                                      String portName);
}

testsJNI.java:

import com.*;
import py4j.GatewayServer;

public class testsJNI
{
    public static void main(String args[])
    {
        testsJNI testApp = new testsJNI();

        // Py4J server
        GatewayServer server = new GatewayServer(testApp);
        server.turnLoggingOff();
        server.start();
    }
}

com_mgr_api_JNI.h (created by using javac -h on mgr_api_JNI.java):

/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class com_mgr_api_JNI */

#ifndef _Included_com_mgr_api_JNI
#define _Included_com_mgr_api_JNI
#ifdef __cplusplus
extern "C" {
#endif
/*
 * Class:     com_mgr_api_JNI
 * Method:    createInstance
 * Signature: (Lcom/mgr_api_types/EDisplayType;Ljava/lang/String;Lcom/mgr_api_types/ECommType;Ljava/lang/String;)V
 */
JNIEXPORT void JNICALL Java_com_mgr_1api_1JNI_createInstance
(JNIEnv *, jobject, jobject, jstring, jobject, jstring);

#ifdef __cplusplus
}
#endif
#endif

com_mgr_api_JNI.cpp:

#include "com_mgr_api_JNI.h"
static manager::CManagerApi* manager = NULL;

JNIEXPORT void JNICALL Java_com_mgr_1api_1JNI_createInstance(
                            JNIEnv *env,
                            jobject thisObj, 
                            jobject displayType,
                            jstring displaySerialNumber,
                            jobject commType,
                            jstring portName)
{
  manager::EDisplayType dType = convertObjectToDisplayType(env, displayType);
  const char* serialNumber = env->GetStringUTFChars(displaySerialNumber, 0);
  manager::ECommType comm = convertObjectToCommType(env, commType);
  const char* port = env->GetStringUTFChars(portName, 0);

  char buf[100];
  sprintf(buf,"%s",port);
  std::string portStr = buf;

  sprintf(buf,"%s",serialNumber);
  std::string serialNumStr = buf;

  if (manager == NULL)
  {
    manager = manager::CManagerApi::get();
    manager->initialize(dType, serialNumStr, comm, portStr);
  }

  // Release memory
  env->ReleaseStringUTFChars(displaySerialNumber, serialNumber);
  env->ReleaseStringUTFChars(portName, port);
}

Command line execution of Java code:

java -cp /mnt/c/Workspace/library/java/:.:/home/fred/.local/share/py4j/py4j0.10.7.jar -Djava.library.path=/mnt/c/Workspace/build/library/ testsJNI

Doing a -XshowSettings:properties shows the following properties:

awt.toolkit = sun.awt.X11.XToolkit
file.encoding = UTF-8
file.separator = /
java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment
java.awt.printerjob = sun.print.PSPrinterJob
java.class.path = /mnt/c/Workspace/library/java/
    .
    /home/fred/.local/share/py4j/py4j0.10.7.jar
java.class.version = 55.0
java.home = /usr/lib/jvm/java-11-openjdk-amd64
java.io.tmpdir = /tmp
java.library.path = /mnt/c/Workspace/build/library/

Attempts to solve the problem

Making sure I have a valid native library

Doing an ls on the java.library.path /mnt/c/Workspace/build/library/ shows the library libMngr.so.

If I delete libMngr.so from that location and then try to run Java complains it can't find the library.

Doing the nm command on libMngr.so shows the following:

000000000021f269 T Java_com_mgr_1api_1JNI_createInstance

So Java can find the native library and it has the symbol of the function.

Doing the objdump command:

$objdump -f build/library/libMngr.so

build/library/libMngr.so:     file format elf64-x86-64
architecture: i386:x86-64, flags 0x00000150:
HAS_SYMS, DYNAMIC, D_PAGED
start address 0x000000000018aee0

Shows that the native library is 64 bit and according to the -XshowSettings:properties I am using 64 bit Java.

I have even put a print statement right before System.loadLibrary("Mngr"); in mgr_api_JNI.java to make sure that the native library is loaded only once.

Update

I have regenerated the header file from mgr_api_JNI.java and copied the function declaration to the .cpp file to make sure the function name is correct. But I still get the same error.

Making sure I have valid Java classes

If I do an ls on the java.class.path /mnt/c/Workspace/library/java/ I find all of the Java classes from compiling mgr_api_JNI.java.

If I delete the classes and try to run Java, then Java complains it can't find the classes.

Doing a grep -r createInstance on the java.class.path /mnt/c/Workspace/library/java/ returns:

Binary file com/mgr_api_JNI.class matches
com/mgr_api_JNI.java: public native void createInstance(com.mgr_api_types.EDisplayType displayType,

So Java can find the the compiled Java class and it has the createInstance method in it.

Questions

It appears to me that Java can find all of the needed classes and the native library, but I am still getting the UnsatisfiedLinkError error.

Why am I still getting this error?

What do I need to do to help Java find/recognize the createInstance method?

@Elliot Frisch I don't think that is correct. Everything I have read has indicated that `java.library.path` is just like `LD_LIBRARY_PATH` in Linux. It just provides a list of directories for Java to search to find libraries. Also, if I had to specify the library then Java would have returned an error telling me it couldn't find the library. — Fred, Jun 08 '19 at 16:17
You're also loading `Mngr` but have a library file named `libMgr.so`, not sure if it's just a typo in the question... — Jorn Vernee, Jun 08 '19 at 16:43
Yeah, that was a typo on my part. It is really `libMngr.so`. — Fred, Jun 09 '19 at 00:00

Oo.oO · Accepted Answer · 2019-06-08T19:47:11.007

This is your error

You have library file: /mnt/c/Workspace/build/library/libMgr.so but you load this one in your code: System.loadLibrary("Mngr"); - you have a typo

You can make sure that in case of correct name it works as expected

When you "leave" initial JVM, -Djava.library.path is no longer a valid place to provide info to other JVM. I don't know the details of py4j.GatewayConnection, but it's good to make sure you don't run another JVM instance or, that you don't use JNI there.

I suggest following tests:

set the LD_LIBRARY_PATH to location where your lib is
make sure you can access library in the code with only public class mgr_api_JNI (without Python bridge engine)
if there is a chance that you second JVM is loading native code (and can't find it), try to use _JAVA_OPTIONS=-Djava.library.path=[path to library]. This way, all the JVMs will "see" the location - but you should do that just for testing purposes

It definitely looks like issue with loading the lib from another process. If I reduce your code to something like this:

.
|-- Makefile
|-- README.md
|-- c
|   `-- com_mgr_api_JNI.cc
|-- java
|   `-- com
|       |-- mgr_api_JNI.java
|       `-- mgr_api_types
|           |-- ECommType.java
|           `-- EDisplayType.java
|-- lib
`-- target

and the code itself

com_mgr_api_JNI.cc

#include <iostream>
#include "jni.h"
#include "com_mgr_api_JNI.h"

using namespace std;

JNIEXPORT void JNICALL Java_com_mgr_1api_1JNI_createInstance
  (JNIEnv *env, jobject cls, jobject a, jstring b, jobject c, jstring d) {
  cout << "Hello";
}

java/com/mgr_api_JNI.java

package com;
import com.mgr_api_types.*;

public class mgr_api_JNI
{
  static {
    try {
      System.loadLibrary("Mngr"); // Use "-Djava.library.path=[path to library]" option to load this library
    } catch (UnsatisfiedLinkError e) {
      System.err.println("Native code library 'Mngr' failed to load.\n" + e);
    }
  }

  public native void createInstance(  com.mgr_api_types.EDisplayType displayType,
                                      String displaySerialNumber,
                                      com.mgr_api_types.ECommType commType,
                                      String portName);

  public static void main(String [] arg) {

    mgr_api_JNI obj = new mgr_api_JNI();
    obj.createInstance(new com.mgr_api_types.EDisplayType(), "", new com.mgr_api_types.ECommType(), "");

  }
}

java/com/mgr_api_types/ECommType.java

package com.mgr_api_types;

public class ECommType { }

cat java/com/mgr_api_types/EDisplayType.java

package com.mgr_api_types;

public class EDisplayType { }

and I build the code

Makefile.common

ARCH=$(shell uname -s | tr '[:upper:]' '[:lower:]')
ifeq ($(ARCH),darwin)
  EXT=dylib
else
  EXT=so
endif

Makefile

include Makefile.common

all: compilejava compilec

compilec:
    c++ -std=c++11 -g -shared -fpic -I${JAVA_HOME}/include -I${JAVA_HOME}/include/$(ARCH) c/com_mgr_api_JNI.cc -o lib/libMngr.$(EXT)

compilejava:
    $(JAVA_HOME)/bin/javac -h c -d target -cp target java/com/mgr_api_types/EDisplayType.java
    $(JAVA_HOME)/bin/javac -h c -d target -cp target java/com/mgr_api_types/ECommType.java
    $(JAVA_HOME)/bin/javac -h c -d target -cp target java/com/mgr_api_JNI.java

test:
    $(JAVA_HOME)/bin/java -Djava.library.path=$(LD_LIBRARY_PATH):./lib -cp target com.mgr_api_JNI

clean:
    -rm -rfv target/*
    -rm c/*.h
    -rm -rf lib/*

it works as expected

make test
/Library/Java/JavaVirtualMachines/jdk-12.0.1.jdk/Contents/Home/bin/java -Djava.library.path=:./lib -cp target com.mgr_api_JNI
Hello

I think you have a situation where you class is called from either another instance of JVM, or from something called via JNI, or from another process and your -Djava.library.path is gone.

Also, try to use System.load("full/path/of/library.so") to make sure you can access the library from JVM.

`libMgr.so` was a typo, it is really `libMngr.so`. But I will look into your suggestions. — Fred, Jun 08 '19 at 23:59
It turned out to be an issue where I had another instance of a JVM still running that hadn't died. Once I killed it the error went away. — Fred, Jun 10 '19 at 13:42
@Fred - good to know this can lead to errors in this particular case. I am pretty sure I will investigate py4j to some extent :) — Oo.oO, Jun 10 '19 at 14:47

score 0 · Answer 2 · answered Jun 08 '19 at 16:23

0

The problem occurs when the runtime can not load the corresponding native function in a shared library.

So, in these cases it's always good to regenerate the header file to see if the function name is correct.

In this case your function name looks suspicious

Java_com_1mgr_1api_1JNI_createInstance

The generated name uses the _1 to encode a literal underscore in the Java class/function name. So, based on the class name mgr_api_JNI the name should be

Java_com_mgr_1api_1JNI_createInstance

instead. i.e. the first _1 is just a plain underscore _, and this is also what I see when generating a header file for a class with that name.

answered Jun 08 '19 at 16:23

Jorn Vernee

31,735
4
76
93

Sorry, but I don't see `Java_com_mgr_1api_1JNI_createInstance` (missing the `_1`) anywhere in my post or code. – Fred Jun 08 '19 at 16:29
Yes, I have regenerated the header file and then copied the function declaration into the `.cpp` file. – Fred Jun 08 '19 at 16:29
@Fred Yes, it's not in your post, and that's the problem. i.e. the extra `_1` between the package name and function name is _incorrect_, and should just be an underscore `_`. – Jorn Vernee Jun 08 '19 at 16:30
Understood. I will look into it. – Fred Jun 08 '19 at 16:32