Android: Linking to prebuilt static libraries

Question

I have compiled some static and shared libraries for Android. Specifically, I have the libraries

libcoinblas.a   libcoinlapack.a   libcoinmetis.a   libcoinmumps.a   libipopt.a
libcoinblas.so  libcoinlapack.so  libcoinmetis.so  libcoinmumps.so  libipopt.so

Furthermore, these libraries are inter-dependent, that is,

Lapack requires Blas
Mumps  requires Blas and Metis
Ipopt  requires Mumps, Metis, and Lapack

The Android project correctly links and runs when using the shared libraries, but fails to build with the static libraries.

In the shared case, I am using the cmake file

cmake_minimum_required(VERSION 3.4.1)

add_library( native-lib
             SHARED
             src/main/cpp/cpp_example.cpp
             src/main/cpp/MyNLP.cpp)

# Add dependent libraries
add_library(blas SHARED IMPORTED)
set_property(TARGET blas PROPERTY IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/libs/${ANDROID_ABI}/libcoinblas.so)

add_library(lapack SHARED IMPORTED)
set_property(TARGET lapack PROPERTY IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/libs/${ANDROID_ABI}/libcoinlapack.so)

add_library(metis SHARED IMPORTED)
set_property(TARGET metis PROPERTY IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/libs/${ANDROID_ABI}/libcoinmetis.so)

add_library(mumps SHARED IMPORTED)
set_property(TARGET mumps PROPERTY IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/libs/${ANDROID_ABI}/libcoinmumps.so)

add_library(ipopt SHARED IMPORTED)
set_property(TARGET ipopt PROPERTY IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/libs/${ANDROID_ABI}/libipopt.so)

# Location of header files
include_directories(${CMAKE_SOURCE_DIR}/libs/include
                    ${CMAKE_SOURCE_DIR}/libs/include/ThirdParty)

target_link_libraries( native-lib

                       blas
                       lapack
                       metis
                       mumps
                       ipopt
                       )

and in the static case

cmake_minimum_required(VERSION 3.4.1)

add_library( native-lib
             SHARED
             src/main/cpp/cpp_example.cpp
             src/main/cpp/MyNLP.cpp)

# Add dependent libraries
add_library(blas STATIC IMPORTED)
set_property(TARGET blas PROPERTY IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/libs/${ANDROID_ABI}/libcoinblas.a)

add_library(lapack STATIC IMPORTED)
set_property(TARGET lapack PROPERTY IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/libs/${ANDROID_ABI}/libcoinlapack.a)

add_library(metis STATIC IMPORTED)
set_property(TARGET metis PROPERTY IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/libs/${ANDROID_ABI}/libcoinmetis.a)

add_library(mumps STATIC IMPORTED)
set_property(TARGET mumps PROPERTY IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/libs/${ANDROID_ABI}/libcoinmumps.a)

add_library(ipopt STATIC IMPORTED)
set_property(TARGET ipopt PROPERTY IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/libs/${ANDROID_ABI}/libipopt.a)

# Location of header files
include_directories(${CMAKE_SOURCE_DIR}/libs/include
                    ${CMAKE_SOURCE_DIR}/libs/include/ThirdParty)

target_link_libraries( native-lib

                       blas
                       lapack
                       metis
                       mumps
                       ipopt
                       )

I presumed that I simply need to change how the libraries are added from

add_library(libxxx SHARED IMPORTED)
set_property(TARGET libxxx PROPERTY ... libxxx.so)

to

add_library(libxxx STATIC IMPORTED)
set_property(TARGET libxxx PROPERTY ... libxxx.a)

But that does not work. Specifically, in the static case, I get a bunch (hundreds) of

undefined reference to xxx

errors. For example,

../../../../libs/arm64-v8a/libipopt.a(IpLapack.o): In function `Ipopt::IpLapackDppsv(int, int, double const*, double*, int, int&)':
IpLapack.cpp:(.text+0x3d4): undefined reference to `dppsv_'

Although the errors are not just due to missing Lapack functions, but also Mumps and others.

EDIT

Looking at the specific failed command, I believe that the libraries were specified in the correct order:

FAILED: cmd.exe /C "cd . && clang++.exe --target=aarch64-none-linux-android --gcc-toolchain=C:/Android/android-sdk/ndk-bundle/toolchains/aarch64-linux-android-4.9/prebuilt/windows-x86_64 --sysroot=sysroot -fPIC -isystem C:/Android/android-sdk/ndk-bundle/sysroot/usr/include/aarch64-linux-android -D__ANDROID_API__=23 -g -DANDROID -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -Wa,--noexecstack -Wformat -Werror=format-security -O0 -fno-limit-debug-info -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libatomic.a --sysroot C:/Android/android-sdk/ndk-bundle/platforms/android-23/arch-arm64 -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Wl,-z,noexecstack -Qunused-arguments -Wl,-z,relro -Wl,-z,now -shared -Wl,-soname,libnative-lib.so -o ........\build\intermediates\cmake\debug\obj\arm64-v8a\libnative-lib.so CMakeFiles/native-lib.dir/src/main/cpp/cpp_example.cpp.o CMakeFiles/native-lib.dir/src/main/cpp/MyNLP.cpp.o libcoinblas.a libcoinlapack.a libcoinmetis.a libcoinmumps.a libipopt.a -latomic -lm "C:/Android/android-sdk/ndk-bundle/sources/cxx-stl/gnu-libstdc++/4.9/libs/arm64-v8a/libgnustl_static.a" && cd ."

Note that I sanitized the paths a little bit above so that they are slightly readable, but towards the end you can see that the libraries are listed in the order

libcoinblas.a libcoinlapack.a libcoinmetis.a libcoinmumps.a libipopt.a

EDIT

I also tried changing the link command from target_link_library to link_library:

link_libraries(native-lib blas lapack metis mumps ipopt)

but this also fails. For some reason, in this case, the link command doesn't even include the libraries that it is supposed to link:

FAILED: cmd.exe /C "cd . && C:\Android\android-sdk\ndk-bundle\toolchains\llvm\prebuilt\windows-x86_64\bin\clang++.exe --target=aarch64-none-linux-android --gcc-toolchain=C:/Android/android-sdk/ndk-bundle/toolchains/aarch64-linux-android-4.9/prebuilt/windows-x86_64 --sysroot=C:/Android/android-sdk/ndk-bundle/sysroot -fPIC -isystem C:/Android/android-sdk/ndk-bundle/sysroot/usr/include/aarch64-linux-android -D__ANDROID_API__=23 -g -DANDROID -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -Wa,--noexecstack -Wformat -Werror=format-security -O0 -fno-limit-debug-info -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libatomic.a --sysroot C:/Android/android-sdk/ndk-bundle/platforms/android-23/arch-arm64 -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Wl,-z,noexecstack -Qunused-arguments -Wl,-z,relro -Wl,-z,now -shared -Wl,-soname,libnative-lib.so -o ........\build\intermediates\cmake\debug\obj\arm64-v8a\libnative-lib.so CMakeFiles/native-lib.dir/src/main/cpp/cpp_example.cpp.o CMakeFiles/native-lib.dir/src/main/cpp/MyNLP.cpp.o -latomic -lm "C:/Android/android-sdk/ndk-bundle/sources/cxx-stl/gnu-libstdc++/4.9/libs/arm64-v8a/libgnustl_static.a" && cd ." CMakeFiles/native-lib.dir/src/main/cpp/cpp_example.cpp.o: In function `Java_io_jeti_ipopt_1static_MainActivity_stringFromJNI':

How did you build `Lapack` as a shared library in the first place? I guess when you did you had to somehow specify its dependency on `Blas`. For the static libraries the linking step has not "happened" yet, so you have to explicitly specify the various dependencies in the CMakeFile. Not 100% sure, but give that a try. — athos, Apr 12 '18 at 08:25
Can you get to see the generated linker line, if so check the libraries are present in the correct order: least dependant 1st. — Richard Critten, Apr 12 '18 at 08:27
I built some Android standalone toolchains with gfortran here: https://github.com/jeti/android_fortran — bremen_matt, Apr 12 '18 at 08:41
Then I built ipopt (and blas, lapack etc.). I put the build script here: https://github.com/jeti/android_ipopt — bremen_matt, Apr 12 '18 at 08:42
To avoid having to run the build scripts yourself, I put the compiled files in the `releases` section of each repo — bremen_matt, Apr 12 '18 at 08:42
So, for instance, in the release section of https://github.com/jeti/android_ipopt, I have the shared and static libraries mentioned above for some common Android architectures. — bremen_matt, Apr 12 '18 at 08:43
As for the topic at hand, I will update the question to show variations that I have tried. — bremen_matt, Apr 12 '18 at 08:45
Where you say `and in the static case` should you not be using **STATIC** *instead* of **SHARED** ? *i.e.* `add_library( native-lib STATIC src/main/cpp/cpp_example.cpp src/main/cpp/MyNLP.cpp)` — Jon Goodwin, Apr 14 '18 at 19:33
I was under the impression that the Android library itself always had to be shared. Anyways, I will test either today or tomorrow with this change... — bremen_matt, Apr 14 '18 at 19:36
Here is a classic case where a minimal example, demonstration the problem, should be helpful to you and us, and should be easy to construct. — Jon Goodwin, Apr 14 '18 at 19:44
@JonGoodwin You are right. It's a shame that questions with bounties cannot be close-voted. I don't get how is upvoting such useless (for other people) questions. — usr1234567, Apr 15 '18 at 09:24

Alex Cohn · Accepted Answer · 2018-05-14T18:47:22.210

8

Your libraries are interdependent:

Lapack requires Blas
Mumps  requires Blas and Metis
Ipopt  requires Mumps, Metis, and Lapack

This means that the order for linking them should be reverse:

ipopt
mumps
metis
lapack
blas

If you don't want to waste your time on figuring out the best order, but rather let the linker to find out (this may slow your builds significantly), you can use

target_link_libraries(native-lib -Wl,--start-group blas lapack metis mumps ipopt -Wl,--end-group).

You can also teach CMake about the dependencies between imported static libs, via IMPORTED_LINK_INTERFACE_LIBRARIES, e.g.

set_target_properties(lapack 
  PROPERTIES IMPORTED_LINK_INTERFACE_LIBRARIES 
  blas)

and so on.

This will translate

target_link_libraries( native-lib
                   blas
                   lapack
                   )

To

clang++ -o libnative-lib.so … libblas.a libnlapack.a libblas.a

edited May 14 '18 at 18:47

answered Apr 15 '18 at 12:45

Alex Cohn

56,089
9
113
307

If CMake knows the dependencies, you don't have to provide the order, CMake will figure it out by itself. – usr1234567 Apr 15 '18 at 20:04
@usr1234567 apparently, this didn't happen for the TS in this case. – Alex Cohn Apr 15 '18 at 20:11
Your answer does not answer the question and is IMHO a bad advice. OP should reduce the question and provide a minimal working example. Then we can work out a proper solution. – usr1234567 Apr 15 '18 at 20:33
1

It seems like ordering was indeed the problem. Now it is complaining about missing gfortran in the toolchain, which makes sense since that is a dependency which I forgot to include, and which is not in the default NDK toolchain. Repacking the libraries with that will take a lot of time, so I am tentatively marking this as the answer. – bremen_matt Apr 16 '18 at 08:15
Maybe I am missing something, but if you could build the **.so** versions of these libs, you must have **libgfortran.a** somewhere, and only add it as a dependency following the same pattern as above. – Alex Cohn Apr 16 '18 at 08:49
Yes, you are correct. For some of the Android architectures, I see one (mips) or several (arm). But for others, I don't see one (x86). I have to revisit my build script for gfortran. Maybe i did something strange there – bremen_matt Apr 17 '18 at 05:12
1

FYI, using start-group/end-group can be pretty bad for build speed. – Dan Albert May 14 '18 at 18:04

score 0 · Answer 2 · answered Apr 16 '18 at 16:52

This might not be anything to do with libraries at all. It could be to do with the way that dppsv () is prototyped.

The source file corresponding to the linker error you quote in your post is here:

https://github.com/coin-or/Ipopt/blob/master/Ipopt/src/LinAlg/IpLapack.cpp

And this contains the following code snippet:

extern "C" {

/** LAPACK Fortran subroutine DPPSV. */
void F77_FUNC(dppsv,DPPSV)(char *uplo, ipfint *n,
                           ipfint *nrhs, const double *A,
                           double *B, ipfint *ldB, ipfint *info);
}

The F77_FUNC macro is evidently designed to map to the function naming convention used by whatever Fortran compiler you happen to be using, see here:

https://github.com/coin-or/Ipopt/blob/master/Ipopt/src/Common/IpoptConfig.h

Now this is not my area of expertise but it could well be that this macro is not doing the right thing in your case. You can run nm on the relevant .o file produced by the Fortran compiler to see what it is generating with your particular build setup. If it's not dppsv_ then you know what's wrong.

Hi Paul, thanks for the input, but there are hundreds of linking errors that reference different libraries. For instance, some errors are complaining of missing Lapack functions, some are missing mumps functions, etc. I think Alex is correct, but I need to generate some more files to verify that he is correct, and that will take a few days — bremen_matt, Apr 16 '18 at 17:48
OK, good luck. An old-school trick which probably still works is to name libraries twice (or more) in the command line passed to the linker. — Paul Sanders, Apr 17 '18 at 05:05
Definitely naming the libs twice works, but CMake is too smart, you need some special ways (described above) to convince it to use this trick. But as long as the static libs don't have circular dependencies, the most natural way to is list them in reverse order. — Alex Cohn, Apr 17 '18 at 08:05

Android: Linking to prebuilt static libraries

EDIT

EDIT

2 Answers2

Linked