Building Protocol Buffers with Standalone NDK toolchain

Question

I'm having trouble building Protocol Buffers for Android using a standalone NDK toolchain created with the $NDK/build/tools/make-standalone-toolchain.sh script. The problem I'm having is this:

libtool: link: g++ -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -O2 -g -DNDEBUG -o protoc main.o  ./.libs/libprotobuf.a ./.libs/libprotoc.a /home/sizeak/AudioFingerprinting/ProtoBuf/jni/src/.libs/libprotobuf.a -lz -lc
./.libs/libprotoc.a(plugin.pb.o): In function `GoogleOnceInit':
/home/sizeak/AudioFingerprinting/ProtoBuf/jni/src/./google/protobuf/stubs/once.h:115: undefined reference to `pthread_once'

Which seems to imply that the pthreads library is missing, however the sysroot was created by the ndk script so should contain everything, right? I read somewhere that it is unnecessary to explicitly link pthreads when using the standalone toolchain. The configure command I used was:

./configure --host=arm-eabi --with-sysroot=$SYSROOT CC=$CC --enable-cross-compile --with-protoc=protoc LIBS="-lc"

Where the environment variables are set up to point at the sysroot created by the script.

I've seen this problem on a number of pages on the net, but no one seems to have posted an answer. Can anyone help?

is ProtoBuf your Android App and are you trying to create a shared library and use it in this android App.Have you mentioned the pthread option in the android.mk file? — Raulp, Apr 28 '12 at 19:21
ProtoBuf was a project for trying to build the library with ndk-build, before I discovered the stand alone toolchain option. — Simon Jackson, Apr 28 '12 at 20:09

score 6 · Accepted Answer · edited May 23 '17 at 12:10

I managed to build it using the ndk-build method, using information found here: How to build protocol buffer by Android NDK

To setup the configure script, I ran these commands:

export CFLAGS="-march=armv7-a -mfloat-abi=softfp -DGOOGLE_PROTOBUF_NO_RTTI"
export CC=arm-linux-androideabi-gcc
export SYSROOT=$NDK/platforms/android-9/arch-arm
./configure --host=arm-eabi --with-sysroot=$SYSROOT CC=$CC --enable-cross-compile --with-protoc=protoc LIBS="-lc"

This is the Application.mk I used:

APP_STL := gnustl_static
APP_ABI := armeabi-v7a armeabi
APP_PROJECT_PATH := ./
APP_BUILD_SCRIPT := ./Android.mk

And the Android.mk used:

# Copyright (C) 2009 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
CC_LITE_SRC_FILES := \
src/google/protobuf/stubs/common.cc                              \
src/google/protobuf/stubs/once.cc                                \
src/google/protobuf/extension_set.cc                             \
src/google/protobuf/generated_message_util.cc                    \
src/google/protobuf/message_lite.cc                              \
src/google/protobuf/repeated_field.cc                            \
src/google/protobuf/wire_format_lite.cc                          \
src/google/protobuf/io/coded_stream.cc                           \
src/google/protobuf/io/zero_copy_stream.cc                       \
src/google/protobuf/io/zero_copy_stream_impl_lite.cc


COMPILER_SRC_FILES :=  \
src/google/protobuf/descriptor.cc \
src/google/protobuf/descriptor.pb.cc \
src/google/protobuf/descriptor_database.cc \
src/google/protobuf/dynamic_message.cc \
src/google/protobuf/extension_set.cc \
src/google/protobuf/extension_set_heavy.cc \
src/google/protobuf/generated_message_reflection.cc \
src/google/protobuf/generated_message_util.cc \
src/google/protobuf/message.cc \
src/google/protobuf/message_lite.cc \
src/google/protobuf/reflection_ops.cc \
src/google/protobuf/repeated_field.cc \
src/google/protobuf/service.cc \
src/google/protobuf/text_format.cc \
src/google/protobuf/unknown_field_set.cc \
src/google/protobuf/wire_format.cc \
src/google/protobuf/wire_format_lite.cc \
src/google/protobuf/compiler/code_generator.cc \
src/google/protobuf/compiler/command_line_interface.cc \
src/google/protobuf/compiler/importer.cc \
src/google/protobuf/compiler/main.cc \
src/google/protobuf/compiler/parser.cc \
src/google/protobuf/compiler/plugin.cc \
src/google/protobuf/compiler/plugin.pb.cc \
src/google/protobuf/compiler/subprocess.cc \
src/google/protobuf/compiler/zip_writer.cc \
src/google/protobuf/compiler/cpp/cpp_enum.cc \
src/google/protobuf/compiler/cpp/cpp_enum_field.cc \
src/google/protobuf/compiler/cpp/cpp_extension.cc \
src/google/protobuf/compiler/cpp/cpp_field.cc \
src/google/protobuf/compiler/cpp/cpp_file.cc \
src/google/protobuf/compiler/cpp/cpp_generator.cc \
src/google/protobuf/compiler/cpp/cpp_helpers.cc \
src/google/protobuf/compiler/cpp/cpp_message.cc \
src/google/protobuf/compiler/cpp/cpp_message_field.cc \
src/google/protobuf/compiler/cpp/cpp_primitive_field.cc \
src/google/protobuf/compiler/cpp/cpp_service.cc \
src/google/protobuf/compiler/cpp/cpp_string_field.cc \
src/google/protobuf/compiler/java/java_enum.cc \
src/google/protobuf/compiler/java/java_enum_field.cc \
src/google/protobuf/compiler/java/java_extension.cc \
src/google/protobuf/compiler/java/java_field.cc \
src/google/protobuf/compiler/java/java_file.cc \
src/google/protobuf/compiler/java/java_generator.cc \
src/google/protobuf/compiler/java/java_helpers.cc \
src/google/protobuf/compiler/java/java_message.cc \
src/google/protobuf/compiler/java/java_message_field.cc \
src/google/protobuf/compiler/java/java_primitive_field.cc \
src/google/protobuf/compiler/java/java_service.cc \
src/google/protobuf/compiler/javamicro/javamicro_enum.cc \
src/google/protobuf/compiler/javamicro/javamicro_enum_field.cc \
src/google/protobuf/compiler/javamicro/javamicro_field.cc \
src/google/protobuf/compiler/javamicro/javamicro_file.cc \
src/google/protobuf/compiler/javamicro/javamicro_generator.cc \
src/google/protobuf/compiler/javamicro/javamicro_helpers.cc \
src/google/protobuf/compiler/javamicro/javamicro_message.cc \
src/google/protobuf/compiler/javamicro/javamicro_message_field.cc \
src/google/protobuf/compiler/javamicro/javamicro_primitive_field.cc \
src/google/protobuf/compiler/python/python_generator.cc \
src/google/protobuf/io/coded_stream.cc \
src/google/protobuf/io/gzip_stream.cc \
src/google/protobuf/io/printer.cc \
src/google/protobuf/io/tokenizer.cc \
src/google/protobuf/io/zero_copy_stream.cc \
src/google/protobuf/io/zero_copy_stream_impl.cc \
src/google/protobuf/io/zero_copy_stream_impl_lite.cc \
src/google/protobuf/stubs/common.cc \
src/google/protobuf/stubs/once.cc \
src/google/protobuf/stubs/structurally_valid.cc \
src/google/protobuf/stubs/strutil.cc \
src/google/protobuf/stubs/substitute.cc

# C++ full library
# =======================================================
#include $(CLEAR_VARS)

LOCAL_MODULE := libprotobuf-lite
LOCAL_MODULE_TAGS := optional

LOCAL_CPP_EXTENSION := .cc

LOCAL_SRC_FILES := \
$(CC_LITE_SRC_FILES)                                             #\
#src/google/protobuf/stubs/strutil.cc                             \
#src/google/protobuf/stubs/substitute.cc                          \
#src/google/protobuf/stubs/structurally_valid.cc                  \
#src/google/protobuf/descriptor.cc                                \
#src/google/protobuf/descriptor.pb.cc                             \
#src/google/protobuf/descriptor_database.cc                       \
#src/google/protobuf/dynamic_message.cc                           \
#src/google/protobuf/extension_set_heavy.cc                       \
#src/google/protobuf/generated_message_reflection.cc              \
#src/google/protobuf/message.cc                                   \
#src/google/protobuf/reflection_ops.cc                            \
#src/google/protobuf/service.cc                                   \
#src/google/protobuf/text_format.cc                               \
#src/google/protobuf/unknown_field_set.cc                         \
#src/google/protobuf/wire_format.cc                               \
#src/google/protobuf/io/gzip_stream.cc                            \
#src/google/protobuf/io/printer.cc                                \
#src/google/protobuf/io/tokenizer.cc                              \
#src/google/protobuf/io/zero_copy_stream_impl.cc                  \
#src/google/protobuf/compiler/importer.cc                         \
#src/google/protobuf/compiler/parser.cc

LOCAL_C_INCLUDES := \
$(LOCAL_PATH)/src

LOCAL_C_INCLUDES := \
$(LOCAL_PATH)/android \
bionic \
$(LOCAL_PATH)/src \
$(JNI_H_INCLUDE)

LOCAL_SHARED_LIBRARIES := \
libz libcutils libutils
LOCAL_LDLIBS := -lz
# stlport conflicts with the host stl library
#ifneq ($(TARGET_SIMULATOR),true)
#LOCAL_C_INCLUDES += external/stlport/stlport
#LOCAL_SHARED_LIBRARIES += libstlport
#endif

# Define the header files to be copied
#LOCAL_COPY_HEADERS := \
#    src/google/protobuf/stubs/once.h \
#    src/google/protobuf/stubs/common.h \
#    src/google/protobuf/io/coded_stream.h \
#    src/google/protobuf/generated_message_util.h \
#    src/google/protobuf/repeated_field.h \
#    src/google/protobuf/extension_set.h \
#    src/google/protobuf/wire_format_lite_inl.h
#
#LOCAL_COPY_HEADERS_TO := $(LOCAL_MODULE)

LOCAL_CFLAGS := -DGOOGLE_PROTOBUF_NO_RTTI

include $(BUILD_SHARED_LIBRARY)

To include the built library in my other NDK project that I'm using Protocol Buffers in, I had to copy the library into the jni folder of the project using the library and modify its Android.mk to add a reference to it:

...
...
LOCAL_LDLIBS := -llog -lOpenSLES -lGLESv2 jni/libprotobuf-lite.so   
include $(BUILD_SHARED_LIBRARY)

include $(CLEAR_VARS)

LOCAL_MODULE := protobuf-lite
LOCAL_SRC_FILES := libprotobuf-lite.so 

include $(PREBUILT_SHARED_LIBRARY)

I think that's it, hope someone finds it useful.

Thanks for Sharing. What this **./Configuration** file contain, you mention to setup configure script. — CoDe, Sep 18 '15 at 10:24
Also while following mention process I'm getting **No rule to make target `jni/src/google/protobuf/stubs/common.cc..'** error. Any suggestion if I'm missing somehting! — CoDe, Sep 18 '15 at 10:46
Ok got it...forgot to add lib source file...now it start build but soon respond failure with message **jni/./src/google/protobuf/stubs/common.cc:51:2: error: #error "No suitable threading library available.** Any suggestion! — CoDe, Sep 18 '15 at 11:33
./configure will run the configure script that is in the current directory, the instructions assumed you had your terminal open in the root of the Protocol Buffers source. You need to configure the build before you can run ndk-build because it is needed to configure some auto generated headers etc. in the protocol buffers source. That is probably why you are getting the error about the missing threading library. I hope that is what you were asking! — Simon Jackson, Sep 18 '15 at 14:17
Thanks Simon..it work great. But when I include Proto.h file, it respond back with following error "google/protobuf/stubs/common.h". Also when I built source file it generate **libgnustl_shared.so** instead **libprotobuf-lite.so** . Am I missing something here, please suggest. — CoDe, Sep 22 '15 at 10:55
On ndk-build clean it giving log **[armeabi-v7a] Clean : protobuf-lite [armeabi-v7a]** . but as mentioned lib/ directory doesn't show protobuf-lite.so file there. Am I missing something. — CoDe, Sep 22 '15 at 11:07
@Shubh were you able to solve the "no suitable threading library available" issue? I'm seeing the same thing, inspite of running configure with the options above. — Sagar, Dec 01 '15 at 21:41
Yes ..that was done..But i need to check as there was some lib configuration that I have to done that time. — CoDe, Dec 02 '15 at 01:32
I am also having the "/src/google/protobuf/stubs/common.cc:52:2: error: #error "No suitable threading library available." I get this error in all of my attempts. I tried various methods. (note: I am on windows) Any suggestions? — MatheuGrondin, Dec 22 '18 at 16:16

score 0 · Answer 2 · answered Jan 02 '19 at 16:56

I had a hard time building Protobuf for android using ndk-build on windows but managed to do it using: - Cygwin - android-ndk-r12b

These steps are for protobuf 3.6.1. Steps: - Clone protobuf repo - At the root of the repo, create a new folder called "jni" (/path/to/repo/protobuf/jni) - Create a new file called Android.mk ad paste the following lines inside

# Copyright (C) 2009 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#

LOCAL_PATH := $(call my-dir)

IGNORED_WARNINGS := -Wno-sign-compare -Wno-unused-parameter -Wno-sign-promo -Wno-error=return-type

COMPILER_SRC_FILES := \
    ../src/google/protobuf/arena.cc \
    ../src/google/protobuf/arenastring.cc \
    ../src/google/protobuf/extension_set.cc \
    ../src/google/protobuf/generated_message_table_driven_lite.cc \
    ../src/google/protobuf/generated_message_util.cc \
    ../src/google/protobuf/implicit_weak_message.cc \
    ../src/google/protobuf/io/coded_stream.cc \
    ../src/google/protobuf/io/zero_copy_stream.cc \
    ../src/google/protobuf/io/zero_copy_stream_impl_lite.cc \
    ../src/google/protobuf/message_lite.cc \
    ../src/google/protobuf/repeated_field.cc \
    ../src/google/protobuf/stubs/bytestream.cc \
    ../src/google/protobuf/stubs/common.cc \
    ../src/google/protobuf/stubs/int128.cc \
    ../src/google/protobuf/stubs/io_win32.cc \
    ../src/google/protobuf/stubs/status.cc \
    ../src/google/protobuf/stubs/statusor.cc \
    ../src/google/protobuf/stubs/stringpiece.cc \
    ../src/google/protobuf/stubs/stringprintf.cc \
    ../src/google/protobuf/stubs/structurally_valid.cc \
    ../src/google/protobuf/stubs/strutil.cc \
    ../src/google/protobuf/stubs/time.cc \
    ../src/google/protobuf/wire_format_lite.cc \
    ../src/google/protobuf/any.cc \
    ../src/google/protobuf/any.pb.cc \
    ../src/google/protobuf/api.pb.cc \
    ../src/google/protobuf/compiler/importer.cc \
    ../src/google/protobuf/compiler/parser.cc \
    ../src/google/protobuf/descriptor.cc \
    ../src/google/protobuf/descriptor.pb.cc \
    ../src/google/protobuf/descriptor_database.cc \
    ../src/google/protobuf/duration.pb.cc \
    ../src/google/protobuf/dynamic_message.cc \
    ../src/google/protobuf/empty.pb.cc \
    ../src/google/protobuf/extension_set_heavy.cc \
    ../src/google/protobuf/field_mask.pb.cc \
    ../src/google/protobuf/generated_message_reflection.cc \
    ../src/google/protobuf/generated_message_table_driven.cc \
    ../src/google/protobuf/io/gzip_stream.cc \
    ../src/google/protobuf/io/printer.cc \
    ../src/google/protobuf/io/strtod.cc \
    ../src/google/protobuf/io/tokenizer.cc \
    ../src/google/protobuf/io/zero_copy_stream_impl.cc \
    ../src/google/protobuf/map_field.cc \
    ../src/google/protobuf/message.cc \
    ../src/google/protobuf/reflection_ops.cc \
    ../src/google/protobuf/service.cc \
    ../src/google/protobuf/source_context.pb.cc \
    ../src/google/protobuf/struct.pb.cc \
    ../src/google/protobuf/stubs/mathlimits.cc \
    ../src/google/protobuf/stubs/substitute.cc \
    ../src/google/protobuf/text_format.cc \
    ../src/google/protobuf/timestamp.pb.cc \
    ../src/google/protobuf/type.pb.cc \
    ../src/google/protobuf/unknown_field_set.cc \
    ../src/google/protobuf/util/delimited_message_util.cc \
    ../src/google/protobuf/util/field_comparator.cc \
    ../src/google/protobuf/util/field_mask_util.cc \
    ../src/google/protobuf/util/internal/datapiece.cc \
    ../src/google/protobuf/util/internal/default_value_objectwriter.cc \
    ../src/google/protobuf/util/internal/error_listener.cc \
    ../src/google/protobuf/util/internal/field_mask_utility.cc \
    ../src/google/protobuf/util/internal/json_escaping.cc \
    ../src/google/protobuf/util/internal/json_objectwriter.cc \
    ../src/google/protobuf/util/internal/json_stream_parser.cc \
    ../src/google/protobuf/util/internal/object_writer.cc \
    ../src/google/protobuf/util/internal/proto_writer.cc \
    ../src/google/protobuf/util/internal/protostream_objectsource.cc \
    ../src/google/protobuf/util/internal/protostream_objectwriter.cc \
    ../src/google/protobuf/util/internal/type_info.cc \
    ../src/google/protobuf/util/internal/type_info_test_helper.cc \
    ../src/google/protobuf/util/internal/utility.cc \
    ../src/google/protobuf/util/json_util.cc \
    ../src/google/protobuf/util/message_differencer.cc \
    ../src/google/protobuf/util/time_util.cc \
    ../src/google/protobuf/util/type_resolver_util.cc \
    ../src/google/protobuf/wire_format.cc \
    ../src/google/protobuf/wrappers.pb.cc \

include $(CLEAR_VARS)

LOCAL_MODULE := libprotobuf
LOCAL_CPP_EXTENSION := .cc
LOCAL_SRC_FILES := $(COMPILER_SRC_FILES)

LOCAL_C_INCLUDES := \
    $(LOCAL_PATH)/../android \
    $(LOCAL_PATH)/../src

LOCAL_CFLAGS := -frtti -DHAVE_PTHREAD $(IGNORED_WARNINGS)

LOCAL_EXPORT_C_INCLUDE_DIRS := $(LOCAL_PATH)/../src

LOCAL_CPPFLAGS := -std=c++11
LOCAL_C_INCLUDES += ${ANDROID_NDK}sources/cxx-stl/gnu-libstdc++/4.9/include
LOCAL_C_INCLUDES += ${ANDROID_NDK}sources/cxx-stl/gnu-libstdc++/4.9/libs/$(APP_ABI)/include
LOCAL_C_INCLUDES += ${ANDROID_NDK}platforms/${APP_PLATFORM}/arch-$(TARGET_ARCH)/usr/include
LOCAL_LDLIBS += -lz -llog
LOCAL_STATIC_LIBRARIES += -lpthread
LOCAL_EXPORT_LDLIBS += -lz
LOCAL_EXPORT_CFLAGS := $(LOCAL_CFLAGS)
LOCAL_EXPORT_CPPFLAGS := $(LOCAL_CPPFLAGS)
include $(BUILD_SHARED_LIBRARY)

Create a file called Application.mk and paste the following code inside

APP_STL := gnustl_static
APP_ABI := armeabi-v7a
APP_PLATFORM:=android-27

Launch cygwin and navigate to the protobuf repo.
run ./Configure (if it fails, it may be because you are missing some tools in cygwin. Install them)
Export the ANDROID_NDK environment variable that points to the root of the ndk folder
Add the path to the ndk-build tool in the PATH environment variable
run ndk-build

A "libs" folder should be created with the libprotobuf.so for the architecture you have specified in the application.mk

Hope this helps

Building Protocol Buffers with Standalone NDK toolchain

2 Answers2

Linked