1

I want to learn more about cross-compiling. That's why I decided to start from scratch. I just wrote a tiny "Hello World" program that I'm going to try to cross-compile for my raspberrypi-zero-W:

main.cpp:

#include <cstdio>
int main()
{
  std::puts("Hello World");
}

Therefor that I wanted to start with llvm/clang/lld/libc++ (because clang promotes in the docs that it already is itself a cross-compiler), I checked out the sources of llvm: git clone --single-branch -b release/13.x https://github.com/llvm/llvm-project.git llvm/src and build and installed libcxx, libcxxabi and libunwind

cross compiling libc++ and libc++abi

cd ./llvm/src
mkdir build_libcxx_arm
cmake -G Ninja -DCMAKE_CROSSCOMPILING=True -DLLVM_DEFAULT_TARGET_TRIPLE=arm-linux-gnueabihf -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=../arm -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -B build_libcxx_arm -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libunwind" -S runtimes
cmake --build ./build_libcxx_arm -- -j8
cmake --install ./build_libcxx_arm

Now I felt ready to give it a try to compile my "HelloWorld" with my new libs:

cross-compiling HelloWorld

cd ../..
clang++ -std=c++20 -stdlib=libc++ -O3 -v -fuse-ld=lld --target=arm-linux-gnueabihf -mcpu=arm1176jzf-s -mfloat-abi=hard --sysroot=./llvm/arm/ -o ./build_arm1176jzf-s/hello_clang src/main.cpp

I now expected clang to compile my main.cpp for my raspberry-pi-zero and to link it via ld.lld against my self-build libc++. But unfortunately I now get some linker errors:

error from clang++

clang version 13.0.0
Target: arm-unknown-linux-gnueabihf
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/11.2.0
Selected GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/11.2.0
Candidate multilib: .;@m32
Selected multilib: .;@m32
 "/usr/bin/clang-13" -cc1 -triple armv6kz-unknown-linux-gnueabihf -emit-obj --mrelax-relocations -disable-free -disable-llvm-verifier -discard-value-names -main-file-name main.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=none -fmath-errno -fno-rounding-math -mconstructor-aliases -target-cpu arm1176jzf-s -target-feature -crc -target-feature -dotprod -target-feature +dsp -target-feature -mve -target-feature -mve.fp -target-feature -ras -target-feature -bf16 -target-feature -sb -target-feature -i8mm -target-feature -lob -target-feature -cdecp0 -target-feature -cdecp1 -target-feature -cdecp2 -target-feature -cdecp3 -target-feature -cdecp4 -target-feature -cdecp5 -target-feature -cdecp6 -target-feature -cdecp7 -target-feature -hwdiv-arm -target-feature -hwdiv -target-feature +vfp2 -target-feature +vfp2sp -target-feature -vfp3 -target-feature -vfp3d16 -target-feature -vfp3d16sp -target-feature -vfp3sp -target-feature -fp16 -target-feature -vfp4 -target-feature -vfp4d16 -target-feature -vfp4d16sp -target-feature -vfp4sp -target-feature -fp-armv8 -target-feature -fp-armv8d16 -target-feature -fp-armv8d16sp -target-feature -fp-armv8sp -target-feature -fullfp16 -target-feature +fp64 -target-feature -d32 -target-feature -neon -target-feature -fp16fml -target-feature -crypto -target-feature -sha2 -target-feature -aes -target-feature +strict-align -target-abi aapcs-linux -mfloat-abi hard -fallow-half-arguments-and-returns -debugger-tuning=gdb -v -fcoverage-compilation-dir=/d/trash/cross-learning -resource-dir /usr/lib/clang/13.0.0 -isysroot ./llvm/arm/ -internal-isystem /usr/bin/../include/c++/v1 -internal-isystem /usr/lib/clang/13.0.0/include -internal-isystem ./llvm/arm//usr/local/include -internal-isystem /usr/bin/../lib/gcc/arm-linux-gnueabihf/11.2.0/../../../../arm-linux-gnueabihf/include -internal-externc-isystem ./llvm/arm//include -internal-externc-isystem ./llvm/arm//usr/include -O3 -std=c++20 -fdeprecated-macro -fdebug-compilation-dir=/d/trash/cross-learning -ferror-limit 19 -stack-protector 2 -fno-signed-char -fgnuc-version=4.2.1 -fno-implicit-modules -fcxx-exceptions -fexceptions -fcolor-diagnostics -vectorize-loops -vectorize-slp -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/main-a46033.o -x c++ src/main.cpp
clang -cc1 version 13.0.0 based upon LLVM 13.0.0 default target x86_64-pc-linux-gnu
ignoring nonexistent directory "./llvm/arm//usr/local/include"
ignoring nonexistent directory "./llvm/arm//usr/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/bin/../include/c++/v1
 /usr/lib/clang/13.0.0/include
 /usr/bin/../lib/gcc/arm-linux-gnueabihf/11.2.0/../../../../arm-linux-gnueabihf/include
 ./llvm/arm//include
End of search list.
 "/usr/bin/ld.lld" --sysroot=./llvm/arm/ -pie -EL -X --eh-frame-hdr -m armelf_linux_eabi -dynamic-linker /lib/ld-linux-armhf.so.3 -o ./build_arm1176jzf-s/hello_clang /usr/bin/../lib/gcc/arm-linux-gnueabihf/11.2.0/../../../../arm-linux-gnueabihf/lib/../lib/Scrt1.o /usr/bin/../lib/gcc/arm-linux-gnueabihf/11.2.0/../../../../arm-linux-gnueabihf/lib/../lib/crti.o /usr/bin/../lib/gcc/arm-linux-gnueabihf/11.2.0/crtbeginS.o -L/usr/bin/../lib/gcc/arm-linux-gnueabihf/11.2.0 -L/usr/bin/../lib/gcc/arm-linux-gnueabihf/11.2.0/../../../../arm-linux-gnueabihf/lib/../lib -L./llvm/arm//lib/../lib -L/usr/bin/../lib/gcc/arm-linux-gnueabihf/11.2.0/../../../../arm-linux-gnueabihf/lib -L./llvm/arm//lib /tmp/main-a46033.o -lc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/bin/../lib/gcc/arm-linux-gnueabihf/11.2.0/crtendS.o /usr/bin/../lib/gcc/arm-linux-gnueabihf/11.2.0/../../../../arm-linux-gnueabihf/lib/../lib/crtn.o
ld.lld: error: cannot open /lib/ld-linux-armhf.so.3: No such file or directory
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)

Now I have two questions: Why do I need /lib/ld-inux-armhf.so.3 at link time? (I understand why I'd need this at runtime on the raspberry-pi, but why at link-time?) And how can I force clang, not to use the gnu-toolchain installed in my system. It looks like it currently includes and links many things from the gnu-toolchain, which I definitely don't want at the moment.

As I said, I'm trying to learn how it works, that's why I started from scratch here. I (currently) don't want to get gcc/ld/libstdc++ involved. Everything what I tried so far is my plain understanding from what I've read in the documentation. But obviously I'm missing something. I hope someone out there is able to help me get this running. Thanks in advance

Tom Mekken
  • 1,019
  • 1
  • 12
  • 27
  • try chainging `--target=arm` to `--target=arm-linux-gnueabihf`. `arm` is just a generic ARM architecture target and it can't make assumptions about the operating system – S.S. Anne Jan 14 '22 at 16:00
  • thank you. I edited my question according your friendly suggestion. It got me rid of one of the two errors. But now it seems, that some parts of the gnu-toolchain are being used (`/usr/bin/../lib/gcc/arm-linux-gnueabihf/11.2.0/../../../../arm-linux-gnueabihf/include`) – Tom Mekken Jan 14 '22 at 16:31
  • 1
    I again updated the question, cause I got rid of the compiler-errors by recompiling libc++ again with a different command. – Tom Mekken Jan 16 '22 at 12:57
  • I didn't do cross compiling for arm and don't know exact solution but I can suggest you to look these guides: [HowToCrossCompileBuiltinsOnArm](https://llvm.org/docs/HowToCrossCompileBuiltinsOnArm.html), [HowToCrossCompileLLVM](https://llvm.org/docs/HowToCrossCompileLLVM.html), [Cross-compilationUsingClang](https://clang.llvm.org/docs/CrossCompilation.html) and to look cross compiling in `riscv` to get the idea and adapt to `arm` equivalents: [Using Clang to compile for RISC-V](https://stackoverflow.com/questions/68580399/using-clang-to-compile-for-risc-v). – Celuk Jan 18 '22 at 13:57
  • Thanks Celuk for those suggestions. I already read all those articles already. And it works flawlessly if I either use parts of the gnu-toolchain, or if I copy the whole sysroot from my target itself. But what I want to do (for learning purposes) to be fully independent from GNU-toolchain. I'm kind of trying to build my very own sysroot (libc, libcxx, compiler-rt and so on) – Tom Mekken Jan 19 '22 at 07:42
  • It looks like you are using your `pre-installed clang` on your system not github clone (it points installed directory as `InstalledDir: /usr/bin`) that probably not build for `arm`, right? (it says `default target x86_64-pc-linux-gnu`) So, I couldn't understand that you cloned and build library from github but after you used your `pre-installed system clang`. If I don't misunderstand you should use `clang executable` from your github built as `{your-llvm-github-clone-build-directory}/bin/clang++` – Celuk Feb 03 '22 at 06:18
  • For example according to your example: `/home/shc/llvm-project/build/bin/clang++ -std=c++20 -stdlib=libc++ -O3 -v -fuse-ld=lld --target=arm-linux-gnueabihf -mcpu=arm1176jzf-s -mfloat-abi=hard --sysroot=./llvm/arm/ -o ./build_arm1176jzf-s/hello_clang src/main.cpp` – Celuk Feb 03 '22 at 06:18
  • I don't think that you can cross compile your sysroot with pre-installed system clang executable that is not built for arm, so, use it from github clone that is built for arm. – Celuk Feb 03 '22 at 06:23
  • @Celuk, yeah I tried several things. I do have a system-clang and checked out the same version from github and built it for default-target "arm". But I don't get why that should make a difference. Thats only the "default" target to build, but in my understanding clang already *is* a crosscompiler, so that you can build for any target you specify. Anyhow, the clang-doc says the correct way is to bootstrap your build when you want to compile libc++. I did that, that worked, but I'm not able to compile libc and compiler-rt yet – Tom Mekken Feb 04 '22 at 20:27
  • However, while you are using system clang, it is also using your x86_64 system linker too: `/usr/bin/ld.lld` that's why you cannot, I think. Maybe you can force a linker that is for arm: `-fuse-ld={your-arm-linker}` Also you can try to compile your code to assembly code with `-S` flag or to object code with `-c` flag to see intermediate compilation transactions done by `clang`, these should work without errors, but later, the `linker` which is not `clang` gives error. I think `clang` is cross but `linker` not. When you build for arm, linker is built accordingly too, that makes difference. – Celuk Feb 05 '22 at 08:50

0 Answers0