Aim Compile a C++ program on Windows for ARM using only LLVM.
Why LLVM because of permissive licensing.
I'm starting to wonder if my understanding of LLVM is correct.
On the host machine do
- Use clang (front end) to generate intermediate representation. This representation is target independent.
- Use llc (back end) to generate target assembly code.
- Use lld-link.exe to produce executable.
Then execute on the target machine.
Host machine Windows 10, 64bit
Target machine Drive PX with a arm cortex-a57
The program
int main(int argc, char* argv[])
{
int x=41;
x++;
return x;
}
I've checked out and compiled LLVM (using Visual Studio 2015, Release build, CPU= x64)
My attempts
clang.exe -target arm -march=armv8-a -mcpu=cortex-a57 -mfloat-abi=hard -emit-llvm -c -o main.bc main.cpp
llc.exe -march=arm -mcpu=cortex-a57 -mattr=a57,armv8-a,v8 -meabi=gnu -o main.s main.bc
lld-link.exe /entry:main /machine:arm main.s
Error
lld-link.exe: error: main.s: unknown file type
Then I tried doing the front-end steps on Windows and the back-end on the arm machine.
clang.exe -target arm -march=armv8-a -mcpu=cortex-a57 -mfloat-abi=hard -emit-llvm -c -o main.bc main.cpp
llc.exe -march=arm -mcpu=cortex-a57 -mattr=a57,armv8-a,v8 -meabi=gnu -o main.s main.bc
SCP main.s to the arm machine. SSH and
gcc main.s (using gcc as a test. LLVM should do this.)
Error
main.s: Assembler messages:
main.s:2: Error: unknown pseudo-op: `.syntax'
main.s:3: Error: unknown pseudo-op: `.eabi_attribute'
main.s:9: Error: unknown pseudo-op: `.fpu'
main.s:26: Error: junk at end of line, first unrecognized character is `@'
main.s:29: Error: unknown pseudo-op: `.code'
main.s:31: Error: unknown pseudo-op: `.fnstart'
main.s:32: Error: junk at end of line, first unrecognized character is `@'
main.s:34: Error: operand 1 should be an integer register -- `mov r2,#0'
main.s:41: Error: operand 1 should be an integer or stack pointer register -- `add r0,r0,#1'
main.s:45: Error: unknown mnemonic `bx' -- `bx lr'
main.s:48: Error: unknown pseudo-op: `.cantunwind'
main.s:49: Error: unknown pseudo-op: `.fnend'
main.s:50: Error: junk at end of line, first unrecognized character is `@'
So I tried to target only Windows
clang.exe -emit-llvm -c -o main.bc main.cpp
llc.exe -march=x86 -c -o main.s main.bc
ld.lld.exe main.s
Error
ld.lld.exe: error: main.s:1: unknown directive: .text
Then, instead of ld.lld.exe use gcc (Again using gcc as a test. LLVM should do this.)
clang.exe -emit-llvm -c -o main.bc main.cpp
llc.exe -march=x86 -c -o main.s main.bc
gcc main.s -o main.exe
That works. To test I type
main.exe
echo Exit Code is %errorlevel%
Which returns 42
General question
What are the steps to compile a C++ program under Windows targeting an arm CPU using only LLVM (no gcc, nothing downloaded from ARM)?
Specific questions
- Can the tools that come with self-compiled LLVM (e.g. clang.exe, llc.exe, lld.exe) compile an executable on Windows targeting arm? E.g is lld still under development?
- Why does my attempt to compile and link under Windows, targeting Windows fail?
- Where do the header files and libraries (e.g. libstdc++) come from when linking on the host for the target? I suppose I need to get those from the arm machine? Copy them to the host and tell the linker where to find them ? Is that correct?
Update
So I originally tried Cross-compilation using Clang
clang.exe --target=arm --sysroot=c:\code\clang\FromCmdLine main.cpp -v
The result is
clang.exe: error: linker (via gcc) command failed with exit code 1 (use -v to see invocation)
And the details of -v are
"C:\\llvm\\clang.exe" -cc1 -triple armv4t-- -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name main.cpp -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -target-cpu arm7tdmi -target-feature +soft-float -target-feature +soft-float-abi -target-feature -fp-only-sp -target-feature -d16 -target-feature -vfp2 -target-feature -vfp3 -target-feature -fp16 -target-feature -vfp4 -target-feature -fp-armv8 -target-feature -neon -target-feature -crypto -target-feature +strict-align -target-abi aapcs -msoft-float -mfloat-abi soft -fallow-half-arguments-and-returns -dwarf-column-info -debugger-tuning=gdb -v -resource-dir "c:\\llvm\\clang\\7.0.0" -isysroot "c:\\code" -fdeprecated-macro -fdebug-compilation-dir "c:\\code" -ferror-limit 19 -fmessage-length 293 -fno-signed-char -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -o "C:\\Users\\AppData\\Local\\Temp\\main-b17d06.o" -x c++ main.cpp
clang -cc1 version 7.0.0 based upon LLVM 7.0.0svn default target x86_64-pc-win32
ignoring nonexistent directory "c:\code\usr/local/include"
ignoring nonexistent directory "c:\code\usr/include"
#include "..." search starts here:
#include <...> search starts here:
C:\llvm\clang\7.0.0\include
End of search list.
"C:\\MinGW\\bin\\gcc.exe" "--sysroot=c:\\code" -v -o a.out "C:\\Users\\AppData\\Local\\Temp\\main-b17d06.o"
Using built-in specs.
COLLECT_GCC=C:\MinGW\bin\gcc.exe
COLLECT_LTO_WRAPPER=c:/mingw/bin/../libexec/gcc/mingw32/6.3.0/lto-wrapper.exe
Target: mingw32
Configured with: ../src/gcc-6.3.0/configure --build=x86_64-pc-linux-gnu --host=mingw32 --target=mingw32 --with-gmp=/mingw --with-mpfr --with-mpc=/mingw --with-isl=/mingw --prefix=/mingw --disable-win32-registry --with-arch=i586 --with-tune=generic --enable-languages=c,c++,objc,obj-c++,fortran,ada --with-pkgversion='MinGW.org GCC-6.3.0-1' --enable-static --enable-shared --enable-threads --with-dwarf2 --disable-sjlj-exceptions --enable-version-specific-runtime-libs --with-libiconv-prefix=/mingw --with-libintl-prefix=/mingw --enable-libstdcxx-debug --enable-libgomp --disable-libvtv --enable-nls
Thread model: win32
gcc version 6.3.0 (MinGW.org GCC-6.3.0-1)
COMPILER_PATH=c:/mingw/bin/../libexec/gcc/mingw32/6.3.0/;c:/mingw/bin/../libexec/gcc/;c:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../../../mingw32/bin/
LIBRARY_PATH=c:/mingw/bin/../lib/gcc/mingw32/6.3.0/;c:/mingw/bin/../lib/gcc/;c:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../../../mingw32/lib/;c:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../../;c:/code/clang/FromCmdLine/lib/
COLLECT_GCC_OPTIONS='-v' '-o' 'a.out' '-mtune=generic' '-march=i586'
c:/mingw/bin/../libexec/gcc/mingw32/6.3.0/collect2.exe -plugin c:/mingw/bin/../libexec/gcc/mingw32/6.3.0/liblto_plugin-0.dll -plugin-opt=c:/mingw/bin/../libexec/gcc/mingw32/6.3.0/lto-wrapper.exe -plugin-opt=-fresolution=C:\Users\AppData\Local\Temp\ccufvVIA.res -plugin-opt=-pass-through=-lmingw32 -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_eh -plugin-opt=-pass-through=-lmoldname -plugin-opt=-pass-through=-lmingwex -plugin-opt=-pass-through=-lmsvcrt -plugin-opt=-pass-through=-ladvapi32 -plugin-opt=-pass-through=-lshell32 -plugin-opt=-pass-through=-luser32 -plugin-opt=-pass-through=-lkernel32 -plugin-opt=-pass-through=-lmingw32 -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_eh -plugin-opt=-pass-through=-lmoldname -plugin-opt=-pass-through=-lmingwex -plugin-opt=-pass-through=-lmsvcrt --sysroot=c:\code\clang\FromCmdLine -Bdynamic -o a.out c:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../../crt2.o c:/mingw/bin/../lib/gcc/mingw32/6.3.0/crtbegin.o -Lc:/mingw/bin/../lib/gcc/mingw32/6.3.0 -Lc:/mingw/bin/../lib/gcc -Lc:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../../../mingw32/lib -Lc:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../.. -Lc:/code/clang/FromCmdLine/lib C:\Users\AppData\Local\Temp\main-b17d06.o -lmingw32 -lgcc -lgcc_eh -lmoldname -lmingwex -lmsvcrt -ladvapi32 -lshell32 -luser32 -lkernel32 -lmingw32 -lgcc -lgcc_eh -lmoldname -lmingwex -lmsvcrt c:/mingw/bin/../lib/gcc/mingw32/6.3.0/crtend.o
c:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../../../mingw32/bin/ld.exe: C:\Users\AppData\Local\Temp\main-b17d06.o: Relocations in generic ELF (EM: 40)
C:\Users\AppData\Local\Temp\main-b17d06.o: error adding symbols: File in wrong format
Update
This does not fully answer my question but it does help me to progress.
For a better understanding I found crosstool-NG useful, especially their documentation (chapters 1 to 5).
Then I read the cmake cross compiling documentation.
The I wrote a small cmake C++ test.
Helloworld.cpp
#include <iostream>
int main(int argc, char *argv[])
{
std::cout << "Hello World!" << std::endl;
return 0;
}
CMakeLists.txt
cmake_minimum_required(VERSION 2.8.9)
project (hello)
add_executable(hello helloworld.cpp)
Target specific configuration for cmake. This is from 4.
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR arm)
set(CMAKE_SYSROOT /home/user/x-tools/aarch64-unknown-linux-gnueabi/aarch64-unknown-linux-gnueabi/sysroot/)
set(CMAKE_STAGING_PREFIX /home/user/crosscompile/stage)
set(tools /home/user/x-tools/aarch64-unknown-linux-gnueabi)
set(CMAKE_C_COMPILER ${tools}/bin/aarch64-unknown-linux-gnueabi-gcc)
set(CMAKE_CXX_COMPILER ${tools}/bin/aarch64-unknown-linux-gnueabi-g++)
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
And the command line
cmake -DCMAKE_TOOLCHAIN_FILE=../toolchain_file.txt ..
That cross compiles to ARM and the program runs on the ARM machine.
But this does not use LLVM / Clang. To use LLVM I thought of changing the toolchain configuration to use
set(tools /usr/bin)
set(CMAKE_C_COMPILER ${tools}/clang)
set(CMAKE_CXX_COMPILER ${tools}/clang++)
That failed because that bin folder is for the host machine.
I also tried using the AArch64 download from http://releases.llvm.org/download.html. Yes that also did not work.
So in summary this what is required.
- A sysroot folder with the lib and include folders for the target system. Okay there needs to be more in that sysroot folder than lib and include.
- A toolchain (compiler, assembler, linker) for the target system.