2

I am using a number of static pre-built static libraries in my native android application and everything works fine. Now I want to switch one of my static libraries to be .so. I was successfully able to build .so library by replacing BUILD_STATIC_LIBRARY with BUILD_SHARED_LIBRARY in its android.mk and adding required dependencies.

I was also able to build my application by replacing corresponding PREBUILT_STATIC_LIBRARY with PREBUILT_SHARED_LIBRARY in its android.mk. The resulting application now fails to start. I cannot even get to point where debugger attaches to the application.

Besides that what I do not understand is how the build system knows that the function should be imported from the library. My so library should export one function, but I did not declare it as dllexport/import or something. Still there are no unresolved symbols in my application (when I remove my prebuilt library from the list, the unresolved symbol appears as expected).

The other question is that I see there are two .so files generated. One big file in obj/local/$(TARGET_ARCH_ABI) folder and another small one in libs/$(TARGET_ARCH_ABI). When declaring my prebuilt library I reference the second one in libs folder.

I did try to search stackoverflow for answers and found quite a few related posts:

but I do not see how these posts related to my problem since I can successfully build and even link my application.

Community
  • 1
  • 1
Egor
  • 779
  • 5
  • 20
  • 1
    When your app fails to start. Do you see anything in logcat? – fukanchik Apr 11 '15 at 21:45
  • 1
    Indeed, logcat should be your first resort. Connecting a breakpoint debugger (at either java or native level) is only something to bother with when the logs have failed to make the problem evident - especially when (as you are discovering here) the problem seems to occur before you get a chance. Though if you really wanted to, you could breakpoint your program *before* it tries to load the native libraries. – Chris Stratton Apr 12 '15 at 02:07
  • 1
    You are right to point to the smaller ([strip](https://en.m.wikipedia.org/wiki/Strip_(Unix))ped) copy of the prebuilt library. The other one may be invaluable for debugging. – Alex Cohn Apr 12 '15 at 06:52
  • @Alex Cohn I still do not understand how the linker understands that the symbol should be resolved dynamically. The function declaration gives no clue to this. In Windows, you can either `LoadLibrary` explicitly and then `GetProcAddress`, or declare the function as `__declspec(dllimport)` which tells the linker what to do. How does this work on Linux/Android? Do I need to load the library explicitly (in Java or C++)? How the linker recognizes imported symbols? – Egor Apr 12 '15 at 18:59
  • The linker looks at the symbols that you can list with `nm -D`, as explained in an answer below. – Alex Cohn Apr 12 '15 at 19:16
  • It works now, thank you for help! One last thing I would like to understand is why `nm -D` lists all the symbols despite the fact that I specified `LOCAL_CFLAGS += -fvisibility=hidden` in my .so project. – Egor Apr 12 '15 at 20:59
  • @Egor - In general, `nm -D` dumps all symbols it encounters that are reachable to the outside world. Here, that includes those with `t` (private) and `T` (public or default). What the link/loader does is slightly different. It only links against `T` (public or default). So the link/loader is the one that enforces the visibility policy. The Android link/loader may have slightly different behavior, but I would not expect it to be too different. – jww Apr 12 '15 at 21:16

4 Answers4

4

You need to load the libraries in reverse dependency order in the java code. You previously probably have something like this:

System.loadLibrary("mylib");

Now if your prebuilt library (that was previously a static library, now a shared library) is named dependencylib, you need to change the code for loading the libraries into this:

System.loadLibrary("dependencylib");
System.loadLibrary("mylib");

As for your question how the linker can figure it out; when linking libmylib.so, it looks for all undefined symbols in all the other libraries you specified (i.e. in libdependencylib.so, and in libc.so and other system libraries). As long as all undefined symbols are found somewhere, the linker is ok. Then at runtime, when libmylib.so is loaded, it does the same routine again; all undefined symbols are looked up in the list of symbols loaded in the current process. On linux, you normally don't need to manually mark symbols as dllexport as you do on windows - all non-static symbols are exported by default.

mstorsjo
  • 12,983
  • 2
  • 39
  • 62
  • Thank you, that solved my problem! I still have one question: my app is a native application with very few Java code. I started it from NativeTeapot and then worked with c++ code only. There was nothing like `System.loadLibrary("mylib");` in my any java source, so I randomly put `System.loadLibrary("mydependencylib");` into the very beginning of `onCreate()` function of my application. Is there a right place to do this? – Egor Apr 12 '15 at 20:42
  • 1
    If your app is based on NativeActivity, it loads the library differently. In general, the safest place to load JNI library is from **static constructor**, see my updated answer. – Alex Cohn Apr 13 '15 at 05:49
  • 1
    That's probably a good enough solution - the `NativeActivity` base class does the equivalent of `System.loadLibrary` within its `onCreate` method, so as long as you load the dependencies before calling `super.onCreate`, it should be fine, although using a static constructor as Alex suggested is even safer. – mstorsjo Apr 13 '15 at 05:52
  • @mstorsjo it is important to remember that **NativeActivity** does not call `System.loadLibrary()`, see *http://stackoverflow.com/questions/29453755/is-it-necessary-to-call-system-loadlibrary-explicitly-for-accessing-the-native-m* – Alex Cohn Apr 13 '15 at 05:53
  • 1
    "Normally" all of this would be taken care of automatically; `libmylib.so` does have stored that it requires `libmydependencylib.so` to be loaded in order to run, so all info already exists to do this automatically, which it does since Android 5.0 if I remember correctly. The problem on the earlier versions is simply that the dynamic linker doesn't look in the right directory for app-specific libraries (the ones that `System.loadLibrary` checks) when loading transitive dependencies, which has been fixed now as far as I remember. – mstorsjo Apr 13 '15 at 05:54
  • Alex, I said "the equivalent of `System.loadLibrary`" (i.e., `dlopen` in this case). I didn't try to make a point saying that `NativeActivity` explicitly uses `System.loadLibrary`, I was trying to make a point about at what point in time the library specified in `AndroidManifest.xml` gets loaded, i.e. at what point all dependencies need to be present. But I do agree that a static constructor is even safer. – mstorsjo Apr 13 '15 at 05:56
2

There may be two reasons why the app fails to start after the change of STATIC -> SHARED.

  1. The prebuilt library is not installed. With your device connected, run adb ls -l /data/your.package.name/lib/. Do you see the library there?

  2. The prebuilt library is not loaded. In your main Java class, try

    static { System.loadLibrary("prebuiltname"); System.loadLibrary("yourlib"); }

    This is a static constructor, the safest place to load JNI library.

Alex Cohn
  • 56,089
  • 9
  • 113
  • 307
1

If you are on linux you will see exported symbols using nm -D. example nm -D libzip.so:

...
0000000000009dc0 T zip_unchange
0000000000009dd0 T zip_unchange_all
0000000000009e30 T zip_unchange_archive
0000000000009e60 T _zip_unchange_data

If you want to control visibility of your functions use __attribute__ ((visibility ("default"))) and command line -fvisibility=hidden. More information here.

fukanchik
  • 2,811
  • 24
  • 29
  • 1
    Android is a rather odd Linux, but this basically holds, and there are cross versions of nm for the various Android ABIs provided in the NDK distribution. – Chris Stratton Apr 12 '15 at 02:05
  • @fukanchik: nm -D -C dumps all the symbols in the library (including the function I want to import), while nm -g shows nothing. As suggested, I added `-fvisibility=hidden` to `LOCAL_CFLAGS` and declared my export function with `__attribute__((visibility("default")))`. Still, something goes wrong... – Egor Apr 12 '15 at 18:42
  • Do you see anything in logcat? – fukanchik Apr 12 '15 at 18:47
0

Now I want to switch one of my static libraries to be .so. I was successfully able to build .so library by replacing BUILD_STATIC_LIBRARY with BUILD_SHARED_LIBRARY in its android.mk and adding required dependencies.

I don't think you can do it if its a C++ library. From <doc>/CPLUSPLUS-SUPPORT.html:

Please keep in mind that the static library variant of a given C++ runtime SHALL ONLY BE LINKED INTO A SINGLE BINARY for optimal conditions.

What this means is that if your project consists of a single shared library, you can link against, e.g., stlport_static, and everything will work correctly.

On the other hand, if you have two shared libraries in your project (e.g. libfoo.so and libbar.so) which both link against the same static runtime, each one of them will include a copy of the runtime's code in its final binary image. This is problematic because certain global variables used/provided internally by the runtime are duplicated.

This is likely to result in code that doesn't work correctly, for example:

* memory allocated in one library, and freed in the other would leak or even corrupt the heap.
* exceptions raised in libfoo.so cannot be caught in libbar.so (and may simply crash the program).
* the buffering of cout not working properly

This problem also happens if you want to link an executable and a shared library to the same static library.

In other words, if your project requires several shared library modules, then use the shared library variant of your C++ runtime.

From above, it means everything needs to link against the same C++ standard runtime shared object.

jww
  • 97,681
  • 90
  • 411
  • 885
  • Thank you for the answer, but this seems to be not my problem. I might not be clear enough, but I was talking not about the run-time library, but about one of my own libraries. The objects exported from the libfoo.so are used in libbar.so. They communicate through an interface that avoids all the problems you mentioned. My real problem is that I cannot make the two libraries, libfoo.so and libbar.so work together. The app crashes even before in enters android_main(). – Egor Apr 12 '15 at 00:56
  • 3
    The issue raised in this answer would exist if and only if your formerly static (but now shared) libraries are still built against the static version of the C++ runtime library. If they are properly linked against the shared version, this will not be an issue. – Chris Stratton Apr 12 '15 at 02:04
  • Both my shared libraries are linked against static version of C++ runtime library and everything works fine now. I do not see any reason why it should not work as long as you do not delete memory allocated in other library or throw exceptions across libraries. – Egor Apr 12 '15 at 20:51