38

So when compiling tons of source files with GCC one can use -j to use all available cores. But what about the linker? Is there a similar option to speed up linking or does GCC not support multi-threading? In some larger projects it can really take a while ... (... and I hate to wait!)

Edit: Thanks for pointing out that -j is a option for make and not gcc/g++. But this does not answer my question! I would like to know if gcc can use multi threading while linking a program!

Danvil
  • 22,240
  • 19
  • 65
  • 88
  • You might be interested in distcc http://distcc.org/ which will allow you to distribute compiles over several machines in a network. – Jon Nalley Feb 28 '11 at 13:55
  • 6
    @Jon: I am not interested in parallel compilation but in parallel linking! – Danvil Mar 01 '11 at 10:43
  • 6
    Why did this question get downvoted? God knows gnu linker is dog slow and finding some way to make it link faster would only improve the build cycle. – greatwolf Jun 24 '11 at 09:37
  • 4
    Linking is not an obviously parallel task. Note that you can sometimes reduce the work of the linker using visibility attributes and/or the -fvisibility gcc option. – Marc Glisse Jan 24 '13 at 12:46
  • Not about gnu but finally Apple did a lot to speedup their linker and even developed a new library format in 2023. Remebmer that Apple is not using llvm linker but their own – Lothar Jun 29 '23 at 06:29

6 Answers6

14

Try gold, which was developed by Ian Lance Taylor et al. from Google and released as part of GNU binutils package.

From Wikipedia:

The motivation for writing gold was to make a linker that is faster than the GNU linker, especially for large applications coded in C++

I must admit I haven't tried it myself yet but it's mentioned on WebKitGTK project webpage.

For more information see an article written by the author of gold: A New ELF Linker.

More importantly, see the work on incremental / parallel / concurrent linking by Sander Mathijs van Veen titled Concurrent Linking with the GNU Gold Linker and the bibliography therein.

Adam Romanek
  • 1,809
  • 1
  • 19
  • 36
12

lld, the linker developed by the LLVM project, will use multiple cores by default. I have also found it to be about 2x faster than gold running with multiple threads (-Wl,--threads -Wl,--thread-count,xxx)

Martin Richtarsky
  • 619
  • 7
  • 18
6

In 2022 the mold linker is another great option.

It is the fastest linker available for ELF targets and has the most aggressive parallelism.

Martin Richtarsky
  • 619
  • 7
  • 18
5

The -j option you are referring to is handled by make not gcc.

Using make -j n asks make to run the actions in the Makefile with multiple parallel process (Replace n with a number. In the case of make -j 2 it's 2 process).

Make will handle most synchronization tasks well when doing parallel builds.

Pierre-Luc Simard
  • 2,723
  • 19
  • 27
4

At Replacing ld with gold - any experience? I have benchmarked LD vs gold vs LLVM LLD on a minimal synthetic benchmark, and the time results clearly show that both gold and LLVM LLD are capable of parallel linking according to time, e.g.:

nogold:  wall=4.35s user=3.45s system=0.88s 876820kB
gold:    wall=1.35s user=1.72s system=0.46s 739760kB
lld:     wall=0.73s user=1.20s system=0.24s 625208kB

We know that parallel link was used because the wall time was smaller than the user time in both of those CPUs

Those benchmarks also reproduce Martin Richtarsky's 2x faster LLVM LLD link times.

Ciro Santilli OurBigBook.com
  • 347,512
  • 102
  • 1,199
  • 985
0

Here are two options for those on CMake (tested on 3.16.3):

  • Switch to gold linker, enable multi-threading (set to number of system cores):
find_program(GNU_GOLD_PROGRAM gold)
if (GNU_GOLD_PROGRAM)
    include(ProcessorCount)
    ProcessorCount(HOST_PROC_COUNT)
    add_link_options("-fuse-ld=gold;LINKER:--threads,--thread-count=${HOST_PROC_COUNT}")
endif(GNU_GOLD_PROGRAM)
  • Switch to lld linker (works in multi-threading mode by default):
find_program(LLD_PROGRAM lld)
if(LLD_PROGRAM)
    add_link_options("-fuse-ld=lld")
endif(LLD_PROGRAM)

More complete example: here

MartinBG
  • 1,500
  • 13
  • 22