Gradle Android NDK build external non-JNI library (for AAR)

Question

The Android NDK docs are mostly focused on building C/C++ JNI code. What I want to do is build an external dynamic library written in Go that does not use CMake and has no JNI wrapper.

For the question, it is not important that the library is written in Go, just that it is not written in C. cgo is used to generate C bindings for the dynamic (.so) library, and I have to pass CC=<path/to/android/ndk/clang> to it so that it can use the NDK compiler internally. I do not use JNI. Rather, I use JNA to access the C-exported methods in libstuff.so.

Currently, I do the following:

1. Manually build the shared library with cgo, passing the correct Android compiler path to it
2. Copy the library to myapp/src/main/jniLibs/<arch>/libstuff.so
3. Build the project

(I also have to pass CGO_LDFLAGS="-Wl,-soname,libstuff.so" in step 1, otherwise JNA cannot find it...)

Goal

What I want to do is build the shared library automatically when I build the Android Gradle project, passing on the right compiler binary and architecture to the Go compiler and having libstuff.so for the relevant architectures be included in the APK, or actually the AAR as I am building an Android library.

What I've tried

I've tried with the following CMake script:

cmake_minimum_required(VERSION 3.18.1)
project(stuff_project)
include(ExternalProject)

# Test
message("================")
message(ANDROID_ARCH_NAME=${ANDROID_ARCH_NAME})
message("================")

ExternalProject_Add(external_proj
    PREFIX "path/to/go/library"
    SOURCE_DIR "path/to/go/library"
    BUILD_IN_SOURCE 1
    CONFIGURE_COMMAND ""
    BUILD_COMMAND "<command to build go library>"
    INSTALL_COMMAND ""
)

The command in BUILD_COMMAND builds the library for ${ANDROID_ARCH_NAME} / ${ANDROID_LLVM_TRIPLE} using ${ANDROID_C_COMPILER} with extra flags ${CMAKE_C_FLAGS} and ${CMAKE_SHARED_LINKER_FLAGS} and copies it to ${CMAKE_LIBRARY_OUTPUT_DIRECTORY}.

This is inspired by A/B/WireGuard and it works when running cmake and cmake --build from the command line (where I did not use the Android compiler or anything), but when running Gradle with android { ... externalNativeBuild { cmake { version "3.18.1"; path 'CMakeLists.txt' } } } it only prints the test messages above when configuring and never actually builds the project. (Maybe I have to make external_proj a dependent of something? But of what?)

How do I fix this?

Related

The WireGuard Android app also uses a Go library and uses CMake's add_custom_target with a COMMAND to build the library. However, this library has a JNI wrapper and does not use JNA. I also tried the add_custom_target method, but the command is never executed.

Answer 1

There may be a way to get it to work with ExternalProject_Add, but what I ended up doing is using add_custom_target, and adding the name of this target to lib/build.gradle.

Your CMakeLists.txt in the module folder will look a bit like this:

lib/CMakeLists.txt:

cmake_minimum_required(VERSION 3.18.1)
project(my_project) #TODO
set(libname "libmycoollibrary.so") #TODO
set(go_source "${CMAKE_CURRENT_SOURCE_DIR}/../../go_source/") #TODO

# Android -> Go architecture map
set(arch_map_x86    386)
set(arch_map_x86_64 amd64)
set(arch_map_arm    arm)
set(arch_map_arm64  arm64)

set(GOARCH ${arch_map_${ANDROID_ARCH_NAME}})

# --target has to be specified to compiler & linker as e.g. ANDROID_C_COMPILER may just be 'clang' without prefixes
# CGO_CPPFLAGS are concatenated to CGO_CFLAGS and CGO_CXXFLAGS
# Setting SONAME is required for JNA to work
add_custom_target(shared-lib
    WORKING_DIRECTORY ${go_source}
    COMMENT "Building shared library for ${ANDROID_LLVM_TRIPLE}"
    VERBATIM
    COMMAND ${CMAKE_COMMAND} -E env
        CGO_ENABLED=1 GOOS=android GOARCH=${GOARCH}
        CC=${ANDROID_C_COMPILER} CXX=${ANDROID_CXX_COMPILER}
        CGO_CPPFLAGS=--target=${ANDROID_LLVM_TRIPLE} CGO_CFLAGS=${CMAKE_C_FLAGS} CGO_CXXFLAGS=${CMAKE_CXX_FLAGS}
        CGO_LDFLAGS=${CMAKE_SHARED_LINKER_FLAGS}\ --target=${ANDROID_LLVM_TRIPLE}\ -Wl,-soname,${libname}
        go build -buildmode=c-shared -o ${CMAKE_LIBRARY_OUTPUT_DIRECTORY}/${libname}
            cgo_exports.go #TODO
            implementation.go #TODO
)

You'll have to change paths and source code file names indicated by #TODO. I personally used GNU Make to call the go compiler, but for this answer I changed it to call go directly, using cmake -E env to set environment variables in a cross-platform way. However, the advantage of using GNU Make is that the your library will not be rebuilt when the source code has not changed. Of course, go can be substituted by other compilers, as long as you properly adjust the environment variables for this compiler. The -Wl,-soname,${libname} linker flag is important, because without it JNA will not find your library. The code was originally inspired by code for the WireGuard app, but it uses JNI instead of JNA.

Now shared-lib (the name of the custom target in the add_custom_target command) needs to be added as a dependency in the module's build.gradle. Pay attention to externalNativeBuild and defaultConfig.externalNativeBuild:

lib/build.gradle:

plugins {
    id 'com.android.library'  // Build AAR
}

android {
    //TODO... compileSdk, buildTypes, compileOptions, etc.

    defaultConfig {
        //TODO... minSdk, targetSdk, versionCode, versionName

        testInstrumentationRunner 'androidx.test.runner.AndroidJUnitRunner'

        externalNativeBuild {
            cmake {
                // Specify which target we want to build,
                // corresponds to name in add_custom_target CMake command
                targets 'shared-lib'
            }
        }
    }

    externalNativeBuild {
        cmake {
            version '3.18.1'  // See cmake_minimum_required in CMakeLists.txt
            path 'CMakeLists.txt'
        }
    }

    // Do not cache unit test results as the shared library may have changed
    tasks.matching { t -> t.name in ['testDebugUnitTest', 'testReleaseUnitTest'] }.all {
        outputs.upToDateWhen { false }
    }
}

dependencies {  //TODO You may want to update some of these
    implementation 'net.java.dev.jna:jna:5.10.0@aar'

    testImplementation 'net.java.dev.jna:jna:5.10.0'  // Include jnidispatch library in unit tests
    testImplementation 'junit:junit:4.+'

    androidTestImplementation 'androidx.test:runner:1.4.0'
}

This build.gradle is adjusted for building an AAR library to lib/build/outputs/aar, which is useful if you want to include this as a separate 'package' (with a wrapper around the native API) in another app. I also added some code you may need for unit tests and instrumented tests. For unit tests you'll need to build the shared library for your own pc and add it to PATH.

Now you should be able to call JNA's Native.load("mycoollibrary", NativeApi.class) in your Java code, where mycoollibrary corresponds to part of the ${libname} you defined in CMakeLists.txt and NativeApi is the name of the library interface you defined (interface NativeApi extends Library).