3

I am referring to https://llvm.org/docs/GettingStarted.html to build the LLVM from its source code. I am using Ubuntu 18.04.

$ cmake -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo ../llvm
$ cmake --build .

Below are the errors:

[2978/2994] Linking CXX shared library lib/libLTO.so.12git
FAILED: lib/libLTO.so.12git
: && /usr/bin/c++ -fPIC -fPIC -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -ffunction-sections -fdata-sections -O2 -g -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete   -Wl,-rpath-link,/home/kumarpr/Praveer_LLVM/llvm-project/build/./lib  -Wl,-O3 -Wl,--gc-sections  -Wl,--version-script,"/home/kumarpr/Praveer_LLVM/llvm-project/build/tools/lto/LTO.exports" -shared -Wl,-soname,libLTO.so.12git -o lib/libLTO.so.12git tools/lto/CMakeFiles/LTO.dir/LTODisassembler.cpp.o tools/lto/CMakeFiles/LTO.dir/lto.cpp.o  -Wl,-rpath,::::::::::::::  lib/libLLVMAArch64AsmParser.a  lib/libLLVMAMDGPUAsmParser.a  lib/libLLVMARMAsmParser.a  lib/libLLVMAVRAsmParser.a  lib/libLLVMBPFAsmParser.a  lib/libLLVMHexagonAsmParser.a  lib/libLLVMLanaiAsmParser.a  lib/libLLVMMipsAsmParser.a  lib/libLLVMMSP430AsmParser.a  lib/libLLVMPowerPCAsmParser.a  lib/libLLVMRISCVAsmParser.a  lib/libLLVMSparcAsmParser.a  lib/libLLVMSystemZAsmParser.a  lib/libLLVMWebAssemblyAsmParser.a  lib/libLLVMX86AsmParser.a  lib/libLLVMAArch64CodeGen.a  lib/libLLVMAMDGPUCodeGen.a  lib/libLLVMARMCodeGen.a  lib/libLLVMAVRCodeGen.a  lib/libLLVMBPFCodeGen.a  lib/libLLVMHexagonCodeGen.a  lib/libLLVMLanaiCodeGen.a  lib/libLLVMMipsCodeGen.a  lib/libLLVMMSP430CodeGen.a  lib/libLLVMNVPTXCodeGen.a  lib/libLLVMPowerPCCodeGen.a  lib/libLLVMRISCVCodeGen.a  lib/libLLVMSparcCodeGen.a  lib/libLLVMSystemZCodeGen.a  lib/libLLVMWebAssemblyCodeGen.a  lib/libLLVMX86CodeGen.a  lib/libLLVMXCoreCodeGen.a  lib/libLLVMAArch64Desc.a  lib/libLLVMAMDGPUDesc.a  lib/libLLVMARMDesc.a  lib/libLLVMAVRDesc.a  lib/libLLVMBPFDesc.a  lib/libLLVMHexagonDesc.a  lib/libLLVMLanaiDesc.a  lib/libLLVMMipsDesc.a  lib/libLLVMMSP430Desc.a  lib/libLLVMNVPTXDesc.a  lib/libLLVMPowerPCDesc.a  lib/libLLVMRISCVDesc.a  lib/libLLVMSparcDesc.a  lib/libLLVMSystemZDesc.a  lib/libLLVMWebAssemblyDesc.a  lib/libLLVMX86Desc.a  lib/libLLVMXCoreDesc.a  lib/libLLVMAArch64Disassembler.a  lib/libLLVMAMDGPUDisassembler.a  lib/libLLVMARMDisassembler.a  lib/libLLVMAVRDisassembler.a  lib/libLLVMBPFDisassembler.a  lib/libLLVMHexagonDisassembler.a  lib/libLLVMLanaiDisassembler.a  lib/libLLVMMipsDisassembler.a  lib/libLLVMMSP430Disassembler.a  lib/libLLVMPowerPCDisassembler.a  lib/libLLVMRISCVDisassembler.a  lib/libLLVMSparcDisassembler.a  lib/libLLVMSystemZDisassembler.a  lib/libLLVMWebAssemblyDisassembler.a  lib/libLLVMX86Disassembler.a  lib/libLLVMXCoreDisassembler.a  lib/libLLVMAArch64Info.a  lib/libLLVMAMDGPUInfo.a  lib/libLLVMARMInfo.a  lib/libLLVMAVRInfo.a  lib/libLLVMBPFInfo.a  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiInfo.a  lib/libLLVMMipsInfo.a  lib/libLLVMMSP430Info.a  lib/libLLVMNVPTXInfo.a  lib/libLLVMPowerPCInfo.a  lib/libLLVMRISCVInfo.a  lib/libLLVMSparcInfo.a  lib/libLLVMSystemZInfo.a  lib/libLLVMWebAssemblyInfo.a  lib/libLLVMX86Info.a  lib/libLLVMXCoreInfo.a  lib/libLLVMBitReader.a  lib/libLLVMCore.a  lib/libLLVMCodeGen.a  lib/libLLVMLTO.a  lib/libLLVMMC.a  lib/libLLVMMCDisassembler.a  lib/libLLVMSupport.a  lib/libLLVMTarget.a  lib/libLLVMMIRParser.a  lib/libLLVMHexagonAsmParser.a  lib/libLLVMLanaiAsmParser.a  lib/libLLVMRISCVUtils.a  lib/libLLVMGlobalISel.a  lib/libLLVMCFGuard.a  lib/libLLVMAsmPrinter.a  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMSelectionDAG.a  lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Info.a  lib/libLLVMAArch64Utils.a  lib/libLLVMAMDGPUDesc.a  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMDesc.a  lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a  lib/libLLVMHexagonDesc.a  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiDesc.a  lib/libLLVMLanaiInfo.a  lib/libLLVMSystemZDesc.a  lib/libLLVMSystemZInfo.a  lib/libLLVMWebAssemblyDesc.a  lib/libLLVMWebAssemblyInfo.a  lib/libLLVMMCDisassembler.a  lib/libLLVMCodeGen.a  lib/libLLVMPasses.a  lib/libLLVMTarget.a  lib/libLLVMCoroutines.a  lib/libLLVMipo.a  lib/libLLVMScalarOpts.a  lib/libLLVMVectorize.a  lib/libLLVMFrontendOpenMP.a  lib/libLLVMIRReader.a  lib/libLLVMAsmParser.a  lib/libLLVMInstrumentation.a  lib/libLLVMHelloNew.a  lib/libLLVMBitWriter.a  lib/libLLVMAggressiveInstCombine.a  lib/libLLVMExtensions.a  lib/libLLVMInstCombine.a  lib/libLLVMLinker.a  lib/libLLVMObjCARCOpts.a  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a  lib/libLLVMProfileData.a  lib/libLLVMObject.a  lib/libLLVMBitReader.a  lib/libLLVMCore.a  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a  lib/libLLVMMCParser.a  lib/libLLVMMC.a  lib/libLLVMDebugInfoCodeView.a  lib/libLLVMDebugInfoMSF.a  lib/libLLVMTextAPI.a  lib/libLLVMBinaryFormat.a  lib/libLLVMSupport.a  -lrt  -ldl  -lpthread  -lm  /usr/lib/x86_64-linux-gnu/libz.so  /usr/lib/x86_64-linux-gnu/libtinfo.so  lib/libLLVMDemangle.a && :
collect2: fatal error: ld terminated with signal 9 [Killed]
compilation terminated.
[2980/2994] Linking CXX executable bin/opt
FAILED: bin/opt
: && /usr/bin/c++ -fPIC -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -ffunction-sections -fdata-sections -O2 -g -DNDEBUG -Wl,--export-dynamic  -Wl,-rpath-link,/home/kumarpr/Praveer_LLVM/llvm-project/build/./lib  -Wl,-O3 tools/opt/CMakeFiles/opt.dir/AnalysisWrappers.cpp.o tools/opt/CMakeFiles/opt.dir/BreakpointPrinter.cpp.o tools/opt/CMakeFiles/opt.dir/GraphPrinters.cpp.o tools/opt/CMakeFiles/opt.dir/NewPMDriver.cpp.o tools/opt/CMakeFiles/opt.dir/PassPrinters.cpp.o tools/opt/CMakeFiles/opt.dir/PrintSCC.cpp.o tools/opt/CMakeFiles/opt.dir/opt.cpp.o -o bin/opt  -Wl,-rpath,::::::::::::::  lib/libLLVMAArch64AsmParser.a  lib/libLLVMAMDGPUAsmParser.a  lib/libLLVMARMAsmParser.a  lib/libLLVMAVRAsmParser.a  lib/libLLVMBPFAsmParser.a  lib/libLLVMHexagonAsmParser.a  lib/libLLVMLanaiAsmParser.a  lib/libLLVMMipsAsmParser.a  lib/libLLVMMSP430AsmParser.a  lib/libLLVMPowerPCAsmParser.a  lib/libLLVMRISCVAsmParser.a  lib/libLLVMSparcAsmParser.a  lib/libLLVMSystemZAsmParser.a  lib/libLLVMWebAssemblyAsmParser.a  lib/libLLVMX86AsmParser.a  lib/libLLVMAArch64CodeGen.a  lib/libLLVMAMDGPUCodeGen.a  lib/libLLVMARMCodeGen.a  lib/libLLVMAVRCodeGen.a  lib/libLLVMBPFCodeGen.a  lib/libLLVMHexagonCodeGen.a  lib/libLLVMLanaiCodeGen.a  lib/libLLVMMipsCodeGen.a  lib/libLLVMMSP430CodeGen.a  lib/libLLVMNVPTXCodeGen.a  lib/libLLVMPowerPCCodeGen.a  lib/libLLVMRISCVCodeGen.a  lib/libLLVMSparcCodeGen.a  lib/libLLVMSystemZCodeGen.a  lib/libLLVMWebAssemblyCodeGen.a  lib/libLLVMX86CodeGen.a  lib/libLLVMXCoreCodeGen.a  lib/libLLVMAArch64Desc.a  lib/libLLVMAMDGPUDesc.a  lib/libLLVMARMDesc.a  lib/libLLVMAVRDesc.a  lib/libLLVMBPFDesc.a  lib/libLLVMHexagonDesc.a  lib/libLLVMLanaiDesc.a  lib/libLLVMMipsDesc.a  lib/libLLVMMSP430Desc.a  lib/libLLVMNVPTXDesc.a  lib/libLLVMPowerPCDesc.a  lib/libLLVMRISCVDesc.a  lib/libLLVMSparcDesc.a  lib/libLLVMSystemZDesc.a  lib/libLLVMWebAssemblyDesc.a  lib/libLLVMX86Desc.a  lib/libLLVMXCoreDesc.a  lib/libLLVMAArch64Info.a  lib/libLLVMAMDGPUInfo.a  lib/libLLVMARMInfo.a  lib/libLLVMAVRInfo.a  lib/libLLVMBPFInfo.a  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiInfo.a  lib/libLLVMMipsInfo.a  lib/libLLVMMSP430Info.a  lib/libLLVMNVPTXInfo.a  lib/libLLVMPowerPCInfo.a  lib/libLLVMRISCVInfo.a  lib/libLLVMSparcInfo.a  lib/libLLVMSystemZInfo.a  lib/libLLVMWebAssemblyInfo.a  lib/libLLVMX86Info.a  lib/libLLVMXCoreInfo.a  lib/libLLVMAggressiveInstCombine.a  lib/libLLVMAnalysis.a  lib/libLLVMAsmParser.a  lib/libLLVMBitWriter.a  lib/libLLVMCodeGen.a  lib/libLLVMCore.a  lib/libLLVMCoroutines.a  lib/libLLVMExtensions.a  lib/libLLVMipo.a  lib/libLLVMIRReader.a  lib/libLLVMInstCombine.a  lib/libLLVMInstrumentation.a  lib/libLLVMMC.a  lib/libLLVMObjCARCOpts.a  lib/libLLVMRemarks.a  lib/libLLVMScalarOpts.a  lib/libLLVMSupport.a  lib/libLLVMTarget.a  lib/libLLVMTransformUtils.a  lib/libLLVMVectorize.a  lib/libLLVMPasses.a  -lpthread  lib/libLLVMAArch64Utils.a  lib/libLLVMCoroutines.a  lib/libLLVMObjCARCOpts.a  lib/libLLVMHelloNew.a  lib/libLLVMAMDGPUUtils.a  lib/libLLVMMIRParser.a  lib/libLLVMARMUtils.a  lib/libLLVMHexagonAsmParser.a  lib/libLLVMHexagonDesc.a  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiAsmParser.a  lib/libLLVMLanaiDesc.a  lib/libLLVMLanaiInfo.a  lib/libLLVMipo.a  lib/libLLVMIRReader.a  lib/libLLVMAsmParser.a  lib/libLLVMInstrumentation.a  lib/libLLVMFrontendOpenMP.a  lib/libLLVMLinker.a  lib/libLLVMVectorize.a  lib/libLLVMRISCVUtils.a  lib/libLLVMMCDisassembler.a  lib/libLLVMGlobalISel.a  lib/libLLVMCFGuard.a  lib/libLLVMAsmPrinter.a  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMSelectionDAG.a  lib/libLLVMCodeGen.a  lib/libLLVMBitWriter.a  lib/libLLVMScalarOpts.a  lib/libLLVMAggressiveInstCombine.a  lib/libLLVMInstCombine.a  lib/libLLVMTarget.a  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a  lib/libLLVMProfileData.a  lib/libLLVMObject.a  lib/libLLVMMCParser.a  lib/libLLVMMC.a  lib/libLLVMDebugInfoCodeView.a  lib/libLLVMDebugInfoMSF.a  lib/libLLVMBitReader.a  lib/libLLVMTextAPI.a  lib/libLLVMCore.a  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a  lib/libLLVMBinaryFormat.a  lib/libLLVMSupport.a  -lrt  -ldl  -lpthread  -lm  /usr/lib/x86_64-linux-gnu/libz.so  /usr/lib/x86_64-linux-gnu/libtinfo.so  lib/libLLVMDemangle.a && :
collect2: fatal error: ld terminated with signal 9 [Killed]
compilation terminated.
[2982/2994] Linking CXX executable bin/llvm-isel-fuzzer
FAILED: bin/llvm-isel-fuzzer

anyone has some idea about it ?

Praveer Kumar
  • 912
  • 1
  • 12
  • 25
  • 1
    Try to build a specific release tag, not the master/main branch. – Lex Li Jan 08 '21 at 17:07
  • just curious to know how will it make difference. https://llvm.org/docs/GettingStarted.html#getting-the-source-code-and-building-llvm Checkout LLVM (including related subprojects like Clang): git clone https://github.com/llvm/llvm-project.git Or, on windows, git clone --config core.autocrlf=false https://github.com/llvm/llvm-project.git – Praveer Kumar Jan 08 '21 at 17:23
  • 1
    It makes differences, because no one guarantees an unstable branch like master to be compiled successfully all the times, but release tags must. – Lex Li Jan 08 '21 at 17:31
  • 3
    You might be running out of memory. – arnt Jan 08 '21 at 17:48
  • No, I am not running out of memory actually :( – Praveer Kumar Jan 08 '21 at 17:50
  • 1
    "I am not running out of memory actually" - how did you figure that out? Try running with `make -j1` - if it succeeds, you _are_ running OOM. – yugr Jan 08 '21 at 22:26
  • I have 64 GB RAM in my machine so I was sure that there should not be memory issue. I tried with `htop` to observe the memory consumption – Praveer Kumar Jan 09 '21 at 14:32
  • Yes, you are right it due to low memory . thanks a lot – Praveer Kumar Jan 09 '21 at 15:01

1 Answers1

6

As mentioned in comments you are most likely running out of memory: by default all executables are linked statically so lds use a lot of RAM. There are several ways to counteract this:

  1. Reduce link parallelism via -DLLVM_PARALLEL_LINK_JOBS=1 to avoid starting too many links in parallel (BTW for generic codebase one could use ld-limiter to achieve the same).

  2. Reduce consumed memory by using either or both of -Wl,-no-keep-memory and -Wl,--reduce-memory-overheads linker flags (add them to CMAKE_EXE_LINKER_FLAGS).

  3. Switch to Gold (via -fuse-ld=gold) or lld (via -fuse-ld=lld) linkers (add switch to CMAKE_EXE_LINKER_FLAGS).

  4. In case you plan to frequently rebuild Clang (e.g. for debugging), you may use -DBUILD_SHARED_LIBS=ON to use shared, instead of static, links. You'll no longer have OOMs and also incremental Clang builds are sped up by 100x (at the cost of 2-3x slower Clang runtimes).

yugr
  • 19,769
  • 3
  • 51
  • 96
  • 2
    Now I use below command : `"cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_PARALLEL_LINK_JOBS=1 ../llvm"` `make --build .` I do not see any error, but it seems it does not build everything, the build hardly take 10 mins to finish. below is the message on the terminal: `[2994/2994] Linking CXX shared module unittests/Passes/TestPlugin.so` Is there anything wrong. I am building LLVM first time from its source. SO I do not have idea whether its expected or not – Praveer Kumar Jan 09 '21 at 18:31
  • @PraveerKumar "it seems it does not build everything" - is there any particular executable missing in bin/? – yugr Jan 09 '21 at 19:41
  • I can see that it has generated 83 files in the bin folder. Is there anyway to know the missing files. – Praveer Kumar Jan 11 '21 at 04:29
  • I got the libs files which I was looking for. Thanks a lot – Praveer Kumar Jan 11 '21 at 11:49
  • 1
    Tips for partial parallel build, first build with make -j options ```make -j$(nproc)```, then it compiles to objects files but fails at linking stage. Anyway, we build objects file in parallle. Next build with make without -j options for avoiding linking fail by out of memory, ```make ``` then linking stage run in serial. – Hongsoog Jul 23 '21 at 08:39
  • @yugar, Thanks, with your the second solution, I have build llvm an clang version 11.0.0. on ubuntu 18.04. – Hongsoog Jul 23 '21 at 11:06