2

I'm trying to cross compile Python 3.7 for Android. I see in my output that bz2 if failing with the following error

building '_bz2' extension
/home/dematic/SPE/python3-android/sdk/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -isystem /home/dematic/SPE/python3-android/build/19c-22-aarch64-linux-androideabi-4.9/include -isystem /home/dematic/SPE/python3-android/build/19c-22-aarch64-linux-androideabi-4.9/include/openssl -no-integrated-as -I. -I./Include -target aarch64-none-linux-androideabi22 -target aarch64-none-linux-androideabi22 -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -fPIC -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -I./Include -I. -I/home/dematic/SPE/python3-android/src/Python-3.7.3/Include -I/home/dematic/SPE/python3-android/src/Python-3.7.3 -c /home/dematic/SPE/python3-android/src/Python-3.7.3/Modules/_bz2module.c -o build/temp.linux-aarch64-3.7/home/dematic/SPE/python3-android/src/Python-3.7.3/Modules/_bz2module.o
/home/dematic/SPE/python3-android/sdk/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -isystem /home/dematic/SPE/python3-android/build/19c-22-aarch64-linux-androideabi-4.9/include -isystem /home/dematic/SPE/python3-android/build/19c-22-aarch64-linux-androideabi-4.9/include/openssl -no-integrated-as -shared -target aarch64-none-linux-androideabi22 -fuse-ld=lld -L /home/dematic/SPE/python3-android/build/19c-22-aarch64-linux-androideabi-4.9/lib -target aarch64-none-linux-androideabi22 -fuse-ld=lld -L /home/dematic/SPE/python3-android/build/19c-22-aarch64-linux-androideabi-4.9/lib -target aarch64-none-linux-androideabi22 -fuse-ld=lld -L /home/dematic/SPE/python3-android/build/19c-22-aarch64-linux-androideabi-4.9/lib -fPIC -target aarch64-none-linux-androideabi22 build/temp.linux-aarch64-3.7/home/dematic/SPE/python3-android/src/Python-3.7.3/Modules/_bz2module.o -L. -L/home/dematic/SPE/python3-android/build/19c-22-aarch64-linux-androideabi-4.9/lib -lbz2 -lpython3.7m -o build/lib.linux-aarch64-3.7/_bz2.cpython-37m.so
ld.lld: error: /home/dematic/SPE/python3-android/build/19c-22-aarch64-linux-androideabi-4.9/lib/libbz2.a(bzlib.o) is incompatible with aarch64linux
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I am building bzip2 1.0.6 without any issues, but I assume I'm not linking to it correctly or some other issue. Is there some sort of other architecture I'm supposed to be building?

This is the project I'm trying to build with https://github.com/GRRedWings/python3-android

jww
  • 97,681
  • 90
  • 411
  • 885
Brian S
  • 3,096
  • 37
  • 55

1 Answers1

3

I'm trying to cross compile Python 3.7 for Android. I see in my output that bz2 if failing with the following error

The Bzip2 makefiles are not written for cross-compiles. They effectively ignore a user's flags like CFLAGS and LDFLAGS. The makefiles actually blows away a user's CFLAGS and sets it to CFLAGS=-Wall -Winline -O2 -g $(BIGFILES). Your flags like -target aarch64-none-linux-androideabi22 are not used.

There are two Makefiles in play. One is called Makefile and it builds the static library, if I recall correctly. The second is Makefile-libbz2_so, and it build the shared object. You need to fix the omissions and apply the fixes to both makefiles.

You should probably use a patched Bzip like bzip2-noloader. It honors a user's CFLAGS, CXXFLAGS, LDFLAGS, etc. The check-in of interest is Commit 34d170f31106.

The makefile recipes in bzip2-noloader look similar to the following. They preserve Seward's original settings in BZIP_CFLAGS. But they also utilize CPPFLAGS and allow a user override in CFLAGS. The override will pickup your flags like -target aarch64-none-linux-androideabi22.

blocksort.o: blocksort.c
    $(CC) $(CPPFLAGS) $(BZIP_CFLAGS) $(CFLAGS) -c blocksort.c

Programs use LDFLAGS as expected:

bzip2: libbz2.a bzip2.o
    $(CC) $(CPPFLAGS) $(BZIP_CFLAGS) $(CFLAGS) $(LDFLAGS) -o bzip2 bzip2.o -L. -lbz2

Finally, the bzip2-noloader fork also honor's PREFIX, DESTDIR, etc. So you can perform staged installs, too.

I am building bzip2 1.0.6 without any issues ...

You are probably building for i686 or x86_64, and not Aarch64. The problem does not surface until link time. You can use objdump to inspect the object files, if interested.

Also note the makefile does this:

CC=gcc
AR=ar
RANLIB=ranlib
LDFLAGS=

You may need to tweak those variable assignments, too. Sometimes ar and ranlib use unusual names, like ranlib-5.0. And also be sure the tools are on-path.

The way to write makefiles to avoid these sorts of problems is detailed at 7.2.3 Variables for Specifying Commands in the GNU Coding Standards. The short of it is, (1) leave CFLAGS (and friends) for the user; and (2) if a flag is needed, then always supply it.

The GNU Coding Standards uses this as an example:

CFLAGS = -g
ALL_CFLAGS = -I. $(CFLAGS)
.c.o:
    $(CC) -c $(CPPFLAGS) $(ALL_CFLAGS) $<

Users can override the default CFLAGS of -g, and -I is always added because it is needed for the compile.

jww
  • 97,681
  • 90
  • 411
  • 885
  • In the patch file I have attached here, a lot of the items you have brought out have been handled. https://github.com/GRRedWings/python3-android/blob/master/mk/bzip2/1.0.6/bzip2-1.0.6-makefile-env.patch I guess I need to look at the objdump you reference and see if I can figure out if it's true I'm building 64bit. – Brian S May 30 '19 at 12:56
  • in running the objdump it is saying architeecture is i386:x86-64 – Brian S May 30 '19 at 13:02
  • @GREnvoy - It looks like the Python build system is hiding output from you. I think you need to see what is behind *`building '_bz2' extension...`*. There has to be more than one object file `_bz2module.o`. With Autoconf you use `V=1`, and with Cmake you use `VERBOSE=1`. I don't know what to use for Python. Here is what I see with modified Makefiles and an ARM dev-board: [Bzip2 ARM build](https://pastebin.com/M6KeUZU8); and an Aarhc64 dev-board: [Bzip2 Aarch64 build](https://pastebin.com/pdYuXfYN). – jww May 30 '19 at 14:40