UnsatisfiedLinkError - Rotate camera image data in Android (for use in OpenCV not just display on screen)

Question

I am developing an android app in portrait mode and I want to output the camera to 2 frames onscreen.

The first frame is the straight output from the camera and comes out with the wrong orientation. I can fix this by using this line:

mCamera.setDisplayOrientation(90);

However for the second frame I want to output what is rendered from an OpenCV function in C++. I have not been able to rotate the image properly. My most recent attempt is to rotate using OpenCV in Java but am getting a "UnsatisfiedLinkError", I have posted the log below. Here is the code I am trying to use to rotate the image (credit to: Rotate camera preview to Portrait Android OpenCV Camera) :

        mYuv = new Mat(PreviewSizeHeight + PreviewSizeHeight/2, PreviewSizeWidth, CvType.CV_8UC1);
        mYuv.put(0, 0, arg0);
        Imgproc.cvtColor(mYuv, mRgba, Imgproc.COLOR_YUV420sp2RGBA, 4);

Here is the function that I am using the above code in:

@Override
public void onPreviewFrame(byte[] arg0, Camera arg1) 
{


    // At preview mode, the frame data will push to here.
    if (imageFormat == ImageFormat.NV21)
    {
        mYuv = new Mat(PreviewSizeHeight + PreviewSizeHeight/2, PreviewSizeWidth, CvType.CV_8UC1);
        mYuv.put(0, 0, arg0);
        Imgproc.cvtColor(mYuv, mRgba, Imgproc.COLOR_YUV420sp2RGBA, 4);

        //We only accept the NV21(YUV420) format.
        if ( !bProcessing )
        {
            //FrameData = arg0;
            mHandler.post(DoImageProcessing);
        }
    }
}

And here is the error I am getting:

    03-20 02:06:34.793: D/OpenCV/StaticHelper(13063): Trying to get library list
03-20 02:06:34.803: E/OpenCV/StaticHelper(13063): OpenCV error: Cannot load info library for OpenCV
03-20 02:06:34.803: D/OpenCV/StaticHelper(13063): Library list: ""
03-20 02:06:34.803: D/OpenCV/StaticHelper(13063): First attempt to load libs
03-20 02:06:34.803: D/OpenCV/StaticHelper(13063): Trying to init OpenCV libs
03-20 02:06:34.803: D/OpenCV/StaticHelper(13063): Trying to load library opencv_java
03-20 02:06:34.803: D/OpenCV/StaticHelper(13063): Cannot load library "opencv_java"
03-20 02:06:34.803: W/System.err(13063): java.lang.UnsatisfiedLinkError: Couldn't load opencv_java from loader dalvik.system.PathClassLoader[dexPath=/data/app/my.project.realtimeSpotDetection-102.apk,libraryPath=/data/app-lib/my.project.realtimeSpotDetection-102]: findLibrary returned null
03-20 02:06:34.803: W/System.err(13063):    at java.lang.Runtime.loadLibrary(Runtime.java:358)
03-20 02:06:34.803: W/System.err(13063):    at java.lang.System.loadLibrary(System.java:526)
03-20 02:06:34.803: W/System.err(13063):    at org.opencv.android.StaticHelper.loadLibrary(StaticHelper.java:64)
03-20 02:06:34.803: W/System.err(13063):    at org.opencv.android.StaticHelper.initOpenCVLibs(StaticHelper.java:95)
03-20 02:06:34.803: W/System.err(13063):    at org.opencv.android.StaticHelper.initOpenCV(StaticHelper.java:39)
03-20 02:06:34.803: W/System.err(13063):    at org.opencv.android.OpenCVLoader.initDebug(OpenCVLoader.java:56)
03-20 02:06:34.803: W/System.err(13063):    at my.project.realtimeSpotDetection.CameraPreview.<clinit>(CameraPreview.java:33)
03-20 02:06:34.803: W/System.err(13063):    at my.project.realtimeSpotDetection.realtimeSpotDetection.onCreate(realtimeSpotDetection.java:77)
03-20 02:06:34.803: W/System.err(13063):    at android.app.Activity.performCreate(Activity.java:5453)
03-20 02:06:34.803: W/System.err(13063):    at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1093)
03-20 02:06:34.803: W/System.err(13063):    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2377)
03-20 02:06:34.803: W/System.err(13063):    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2471)
03-20 02:06:34.803: W/System.err(13063):    at android.app.ActivityThread.access$900(ActivityThread.java:175)
03-20 02:06:34.803: W/System.err(13063):    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1308)
03-20 02:06:34.803: W/System.err(13063):    at android.os.Handler.dispatchMessage(Handler.java:102)
03-20 02:06:34.803: W/System.err(13063):    at android.os.Looper.loop(Looper.java:146)
03-20 02:06:34.803: W/System.err(13063):    at android.app.ActivityThread.main(ActivityThread.java:5602)
03-20 02:06:34.803: W/System.err(13063):    at java.lang.reflect.Method.invokeNative(Native Method)
03-20 02:06:34.803: W/System.err(13063):    at java.lang.reflect.Method.invoke(Method.java:515)
03-20 02:06:34.803: W/System.err(13063):    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1283)
03-20 02:06:34.803: W/System.err(13063):    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1099)
03-20 02:06:34.803: W/System.err(13063):    at dalvik.system.NativeStart.main(Native Method)
03-20 02:06:34.803: D/OpenCV/StaticHelper(13063): First attempt to load libs fails
03-20 02:06:34.803: D/dalvikvm(13063): Trying to load lib /data/app-lib/my.project.realtimeSpotDetection-102/libImageProcessing.so 0x425086a8
03-20 02:06:34.808: W/linker(13063): libImageProcessing.so has text relocations. This is wasting memory and is a security risk. Please fix.
03-20 02:06:34.843: D/dalvikvm(13063): Added shared lib /data/app-lib/my.project.realtimeSpotDetection-102/libImageProcessing.so 0x425086a8
03-20 02:06:34.848: D/dalvikvm(13063): No JNI_OnLoad found in /data/app-lib/my.project.realtimeSpotDetection-102/libImageProcessing.so 0x425086a8, skipping init
03-20 02:06:35.038: D/libEGL(13063): loaded /system/lib/egl/libEGL_mali.so
03-20 02:06:35.083: D/libEGL(13063): loaded /system/lib/egl/libGLESv1_CM_mali.so
03-20 02:06:35.088: D/libEGL(13063): loaded /system/lib/egl/libGLESv2_mali.so
03-20 02:06:35.098: E/(13063): Device driver API match
03-20 02:06:35.098: E/(13063): Device driver API version: 29
03-20 02:06:35.098: E/(13063): User space API version: 29 
03-20 02:06:35.103: E/(13063): mali: REVISION=Linux-r3p2-01rel3 BUILD_DATE=Tue Jul 22 19:59:34 KST 2014 
03-20 02:06:35.233: D/OpenGLRenderer(13063): Enabling debug mode 0
03-20 02:06:36.068: I/dalvikvm-heap(13063): Grow heap (frag case) to 11.507MB for 1036816-byte allocation
03-20 02:06:36.143: W/dalvikvm(13063): No implementation found for native Lorg/opencv/core/Mat;.n_Mat:(III)J
03-20 02:06:36.143: D/AndroidRuntime(13063): Shutting down VM
03-20 02:06:36.143: W/dalvikvm(13063): threadid=1: thread exiting with uncaught exception (group=0x41c28c08)
03-20 02:06:36.163: E/AndroidRuntime(13063): FATAL EXCEPTION: main
03-20 02:06:36.163: E/AndroidRuntime(13063): Process: my.project.realtimeSpotDetection, PID: 13063
03-20 02:06:36.163: E/AndroidRuntime(13063): java.lang.UnsatisfiedLinkError: Native method not found: org.opencv.core.Mat.n_Mat:(III)J
03-20 02:06:36.163: E/AndroidRuntime(13063):    at org.opencv.core.Mat.n_Mat(Native Method)
03-20 02:06:36.163: E/AndroidRuntime(13063):    at org.opencv.core.Mat.<init>(Mat.java:477)
03-20 02:06:36.163: E/AndroidRuntime(13063):    at my.project.realtimeSpotDetection.CameraPreview.onPreviewFrame(CameraPreview.java:77)
03-20 02:06:36.163: E/AndroidRuntime(13063):    at android.hardware.Camera$EventHandler.handleMessage(Camera.java:1016)
03-20 02:06:36.163: E/AndroidRuntime(13063):    at android.os.Handler.dispatchMessage(Handler.java:102)
03-20 02:06:36.163: E/AndroidRuntime(13063):    at android.os.Looper.loop(Looper.java:146)
03-20 02:06:36.163: E/AndroidRuntime(13063):    at android.app.ActivityThread.main(ActivityThread.java:5602)
03-20 02:06:36.163: E/AndroidRuntime(13063):    at java.lang.reflect.Method.invokeNative(Native Method)
03-20 02:06:36.163: E/AndroidRuntime(13063):    at java.lang.reflect.Method.invoke(Method.java:515)
03-20 02:06:36.163: E/AndroidRuntime(13063):    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1283)
03-20 02:06:36.163: E/AndroidRuntime(13063):    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1099)
03-20 02:06:36.163: E/AndroidRuntime(13063):    at dalvik.system.NativeStart.main(Native Method)
03-20 02:06:36.173: I/dalvikvm-heap(13063): Grow heap (frag case) to 12.525MB for 1036816-byte allocation

Android.mk file:

LOCAL_PATH := $(call my-dir)

include $(CLEAR_VARS)

OPENCV_LIB_TYPE:=STATIC
OPENCV_CAMERA_MODULES := on 
OPENCV_INSTALL_MODULES := on

## include ../includeOpenCV.mk
OPENCV_MK_PATH:=/home/android/OpenCV-2.4.9-android-sdk/sdk/native/jni/OpenCV.mk
ifeq ("$(wildcard $(OPENCV_MK_PATH))","")
    #try to load OpenCV.mk from default install location
    include $(TOOLCHAIN_PREBUILT_ROOT)/user/share/OpenCV/OpenCV.mk
else
    include $(OPENCV_MK_PATH)
endif

LOCAL_MODULE    := ImageProcessing
LOCAL_SRC_FILES := ImageProcessing.cpp
LOCAL_LDLIBS +=  -llog -ldl

include $(BUILD_SHARED_LIBRARY)

score 0 · Answer 1 · answered Mar 20 '15 at 01:19

0

Is the opencv_java share library in your libs directory within your Android project? The library can easily be included in your eclipse project and not cause compile errors, but it will not be included in your .apk file unless it is in the libs directory

answered Mar 20 '15 at 01:19

Gary Bak

4,746
4
22
39

What files exactly? I have the OpenCV directory (quite a few files) statically linked in the Android.mk file. Do I need to copy some/all of these files? Let me know if you need more info. – adamst85 Mar 20 '15 at 04:30
Looks to me like the OpenCV offers an Eclipse library project file. If you've included it as a library, Project->Properties->Android and add as library, you should not need to include the library by hand. I do see the libOpenCV_java.so file in the libs directory, that is what it is looking for – Gary Bak Mar 20 '15 at 13:46
I already have OpenCV added as a library in eclipse (directory is OpenCV-2.4.9/sdk/java). I did check the /project/libs directory after building and it does not contain the libOpenCV_java.so file. Is there anything specific I need to do to get this file built? I have added more information in the main post. – adamst85 Mar 22 '15 at 12:40
That file may be failing to load because of a dependency. Are all the .so files, that were in the opencv project in your libs directory? Specifically the opencv core file? – Gary Bak Mar 22 '15 at 14:32
Nope, the only .so file that gets built and is in the libs directory is the native C++ file that is a part of the project. I tried copying the file over but every time I run a clean build it removes it. – adamst85 Mar 23 '15 at 00:18
1

Have you seen this post: http://stackoverflow.com/questions/14693558/unsatisfiedlinkerror-n-mat-while-using-opencv2-4-3-with-android-4-0. he had a failure on the same n_Mat object. – Gary Bak Mar 23 '15 at 01:12
Thanks for your help Gary, I solved this in another way but I know now how to do this in future. – adamst85 Apr 15 '15 at 00:11

score 0 · Answer 2 · answered Apr 15 '15 at 00:09

So I ended up using another method as I was writing native code and used OpenCV in the C++ code block. Not an actual solution but Gary's last comment above is probably the closest to a solution (you need to install the OpenCV app on your phone so it can access the libraries)

UnsatisfiedLinkError - Rotate camera image data in Android (for use in OpenCV not just display on screen)

2 Answers2