37

My Haskell project spends lots of time in Linking dist/build/myapp/myapp ... and also in loading shared libraries when executing TemplateHaskell code.

I suspect this is because ld is slow.

How can I improve link times by switching to the gold linker?

nh2
  • 24,526
  • 11
  • 79
  • 128
  • Besides switching linker you could use `-dynamic` flag. It could easily speedup linking tenfold. – vshabanov Apr 06 '17 at 21:52
  • @vshabanov Is this actually the case? I have tried in the past to use dynamic linking and it did not make things faster for my entire cabal project. But I may have done something wrong so that it used `-dynamic-too`, giving me both static and dynamic libraries. It would be great to have a minimal example project that shows if it really makes it faster. – nh2 Apr 07 '17 at 00:13
  • you should use only `-dynamic` GHC option. `-dynamic-too` links **both** statically (slow, large executables) and dynamically (smaller executables, faster link times), so you don't get any speedup. – vshabanov Apr 07 '17 at 15:18
  • @vshabanov I've tried dynamically linked executables now, they are not a good choice for me, because the startup time of my executable (just for the `--help` text) increased from `160 ms` (static) to `4.5 seconds` due to run-time linking. I will stick with static linking for now. – nh2 Apr 08 '17 at 16:51
  • That's quite strange (both 0.16s and 4.5s for app start). What platform you're on? And do you have enough free RAM? Perhaps long linking you have is a cache issue. And interesting what was linking speedup on your projects when you tried dynamic linking? – vshabanov Apr 09 '17 at 00:28
  • 1
    @vshabanov On Linux. I have about 30 GB free RAM. It's not a cache issue, it happens on subsequent starts; the time it takes is 100% user CPU time spent in the ld-linux dynamic linker/loader. [This page](https://www.iecc.com/linker/linker10.html) agrees with the phenomenon: `The runtime performance costs of dynamic linking are substantial compared to those of static linking`. Note I have 100s of entries in `lld`. Regarding speedup, link time seems to have gone down from 2.5 s per executable to 1.5, but that's not worth the increased startup time for me. – nh2 Apr 10 '17 at 02:00
  • 1
    I found [this useful comment](http://stackoverflow.com/a/1607954/263061) that suggests that dynamic linking speed can be drastically improved by setting `-fvisibility=hidden` and manually exporting all exported symbols. Maybe the dynamic loading startup time could be improved by using that. – nh2 Apr 10 '17 at 02:01

1 Answers1

47

Link 3x faster with gold

Since GHC 7.8, you can tell GHC and cabal (at run time without having to recompile GHC) to link with GNU gold.

You need in your .cabal file:

library:
  ghc-options: -optl-fuse-ld=gold
  ld-options:  -fuse-ld=gold

executable myExecutable
  ghc-options: -optl-fuse-ld=gold
  ld-options:  -fuse-ld=gold

(Note you might want to pass these flags to stack/cabal/Setup.hs on the command line instead of hardcoding them in the .cabal file in order to not reduce the portability of the package.)

For me it's 3.5x faster, bringing down the total link time of a project from 150 seconds to 40 seconds.

Update: Link 10x faster with lld

See https://github.com/nh2/link-with-lld-example for a full example; key parts:

library
  ghc-options: "-pgmP clang" "-pgmc clang" "-pgma clang" "-pgml clang" "-optl-fuse-ld=lld"
  ld-options:  -fuse-ld=lld

executable myExecutable
  ghc-options: "-pgmP clang" "-pgmc clang" "-pgma clang" "-pgml clang"
  ld-options:  -fuse-ld=lld

Comparison of link time for the final executable link times my project:

ld   124 seconds
gold  36 seconds
lld   11 seconds
nh2
  • 24,526
  • 11
  • 79
  • 128
  • 2
    How big is your project, if you don't mind me asking? – jberryman Apr 06 '17 at 02:04
  • 2
    @jberryman Not very big, 150 Haskell modules and 8 executables built from them. But it depends on (and links) a couple native libraries like opencv. With `ld`, each library link takes almost 20 seconds. – nh2 Apr 06 '17 at 12:02
  • 2
    Confirming build speed up by reducing time by `40` seconds on 489 modules and 13 executables project (tests and benchmakrs included). – Shersh Apr 06 '17 at 13:41
  • @SebastianGraf It does not work out of the box with `lld`, at least not with the options I provided, because (at least on my system) ghc links by calling `gcc`, and gcc is [not (yet) compatible with `lld`](https://gcc.gnu.org/ml/gcc-patches/2016-07/msg00155.html). You would have to tell ghc to use the clang toolchain instead of gcc; I haven't tried that yet. Would be very interested to know what the flags are for that though. – nh2 Apr 06 '17 at 14:50
  • Wow, that's awesome! Let's start a GHC proposal to use `lld` by default! – Sebastian Graf Apr 07 '17 at 15:16
  • Will it work without changes to the .cabal if you simply overwrite the /usr/bin/ld symlink to ldd? – unhammer Apr 19 '17 at 10:40
  • 1
    @unhammer It will not necessarily work; `lld` doesn't accept all flags that `ld` accepts, and officially `gcc` (which GHC invokes by default for linking) doesn't support linking with `lld`. (Also you mean `lld`, not `ldd` -- just saying because this typo can actually get confusing, as both are linking related programs). – nh2 Apr 19 '17 at 16:05
  • Thanks for the clarification @nh2 – and yes, I've already confused myself with the typo many times :) – unhammer Apr 20 '17 at 07:13
  • @nh2 it looks like `-fuse-ld=lld` is supported now (at lease as of ghc 10; just tried it. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83243). I wonder how that changes things? Can those `ghc-options` go away now? – jberryman Dec 21 '21 at 23:31
  • 1
    @jberryman I think you mean `gcc 10` instead of `ghc 10`? I will need to check what can be removed now, but haven't had time yet. Also there's already a new, even faster linker available now: [mold](https://www.phoronix.com/scan.php?page=news_item&px=Mold-1.0-Released). – nh2 Dec 27 '21 at 20:46