LLVM 6 Trunk Build on Ubuntu 16.04 not building lld

Question

I am building LLVM 6 on Ubuntu 16.04. I want lld too. But no matter what I do, lld doesn't build/install.

Followed the following instruction from here. Still sifting through first and second :)

1. Read the documentation.

2. Read the documentation.

3. Remember that you were warned twice about reading the documentation.

   In particular, the relative paths specified are important.

4. Checkout LLVM:

   cd where-you-want-llvm-to-live
   svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
   

5. Checkout Clang:

   cd where-you-want-llvm-to-live
   cd llvm/tools
   svn co http://llvm.org/svn/llvm-project/cfe/trunk clang
   

6. Checkout Extra Clang Tools [Optional]:

   cd where-you-want-llvm-to-live
   cd llvm/tools/clang/tools
   svn co http://llvm.org/svn/llvm-project/clang-tools-extra/trunk extra
   

7. Checkout LLD linker [Optional]:

   cd where-you-want-llvm-to-live
   cd llvm/tools
   svn co http://llvm.org/svn/llvm-project/lld/trunk lld
   

Didn't download the other optional packages. Since svn download was not working properly (svn: E000104: Error running context: Connection reset by peer), I downloaded respective zip for LLVM, Clang, clang-extra and lld from their github mirrors. Since, I didn't use svn some renaming of the folders had to be done (for e.g. clang-tools-extra-master to just extra).

Then execute the following, after cding into the parent folder of the llvm source folder,

$ mkdir build; cd build
$ cmake ../llvm
$ make

After waiting for +8 hours, I get all the programs (clang, lli, llc etc.) except lld. I am expecting it to appear in the folder build/bin.

I have also tried the option cmake -DLLVM_ENABLE_PROJECTS=llvm/tools/lld ../llvm. (The form -DLLVM_ENABLE_PROJECTS=lld doesn't work and I found this place discussing about the same ENABLE issue.)

I have already tried things with some tweaking three times. Since it takes so much time to compile, I decided to take your help.

A separate issue: While building, linking phase takes 12 GB RAM + 8.8 GB swap space on my laptop!! Does LLVM building really require >20 GB of ram ? (I had closed all other foreground apps, especially firefox)

Please let me know if any more information is needed.

Answer 1

First of all, I think that the LLVM_ENABLE_PROJECTS option is useful if you use a "flat" source directory layout. In your case, if you performed the checkouts as you wrote, then it's not of much use.

Moreover, what @PaulR wrote about using ninja is a very good suggestion. It's faster and very useful when restarting builds and it spawns as many separate compilation jobs concurrently as it can depending on the nproc without having to specify it explicitly.

For faster build and shorter link times, I'd also suggest enabling a shared libs enabled build using the BUILD_SHARED_LIBS option.

Some general good advice can be found here, which can be summarized as:

1. use clang to build clang (a bootstrapped or second-stage build).
2. use the gold linker instead ld for faster linking times.
3. prefer a build producing shared libraries.

You can skip point 1 for now (although you could install an initial llvm/clang from the system package manager)

Lastly, you could limit the compilation for the specific target backend you currently require using the LLVM_TARGETS_TO_BUILD option.

I've built trunk recently (including lld) with this cmake configuration:

CC=gcc CXX=g++ \
cmake -G Ninja \
  -DCMAKE_EXPORT_COMPILE_COMMANDS=On \
  -DBUILD_SHARED_LIBS=On \

  -DLLVM_ENABLE_ASSERTIONS=On \
  -DLLVM_TARGETS_TO_BUILD="X86" \
  -DLLVM_ENABLE_SPHINX=Off \
  -DLLVM_ENABLE_THREADS=On \
  -DLLVM_INSTALL_UTILS=On \

  -DCMAKE_BUILD_TYPE=Debug \
  [path-to-source-root-dir]

You can add the following flags if you need to build libcxx:

[...]
-DLLVM_ENABLE_LIBCXX=On \
-DLIBCXX_ENABLE_EXCEPTIONS=On \
-DLIBCXX_ENABLE_RTTI=On \
[...]

For a bootstrapped build using another llvm/clang and libc++ you can augment the above command as (having that clang in your $PATH environment variable):

LLVM_TOOLCHAIN_LIB_DIR=$(llvm-config --libdir)

LD_FLAGS=""
LD_FLAGS="${LD_FLAGS} -Wl,-L ${LLVM_TOOLCHAIN_LIB_DIR}"
LD_FLAGS="${LD_FLAGS} -Wl,-rpath-link ${LLVM_TOOLCHAIN_LIB_DIR}"
LD_FLAGS="${LD_FLAGS} -lc++ -lc++abi"

CXX_FLAGS=""
CXX_FLAGS="${CXX_FLAGS} -stdlib=libc++ -pthread"

CC=clang CXX=clang++ \
cmake -G Ninja \
  -DCMAKE_EXPORT_COMPILE_COMMANDS=On \
  -DBUILD_SHARED_LIBS=On \
  -DLLVM_ENABLE_LIBCXX=On \
  -DLLVM_ENABLE_ASSERTIONS=On \
  -DLLVM_TARGETS_TO_BUILD="X86" \
  -DLLVM_ENABLE_SPHINX=Off \
  -DLLVM_ENABLE_THREADS=On \
  -DLLVM_INSTALL_UTILS=On \
  -DLIBCXX_ENABLE_EXCEPTIONS=On \
  -DLIBCXX_ENABLE_RTTI=On \
  -DCMAKE_BUILD_TYPE=Debug \
  -DCMAKE_CXX_FLAGS="${CXX_FLAGS}" \
  -DCMAKE_SHARED_LINKER_FLAGS="${LD_FLAGS}" \
  -DCMAKE_MODULE_LINKER_FLAGS="${LD_FLAGS}" \
  -DCMAKE_EXE_LINKER_FLAGS="${LD_FLAGS}" \
  -DCMAKE_POLICY_DEFAULT_CMP0056=NEW \
  -DCMAKE_POLICY_DEFAULT_CMP0058=NEW \
  [path-to-source-root-dir]

Moreover, another relevant SO question of interest can be found here.

As you wrote, reading the documentation, experimenting and reading the documentation again is the way to go.