3

I have a binary which is linked using a large number of object files with many interdependencies. Whenever I recompile even one of these, I need to link the entire binary.

Do linkers (specifically, those of GCC or Clang) support some method of "differential linking", in which enough information is kept about the inter-relations between all the other linked parts, so that the only work that needs to be done when a single part is recompiled is its relations to the other parts + putting them together in the binary?

Note: I'm mostly interested in C++ but I guess this question generalizes at least to C and probably to other compiled languages.

jotik
  • 17,044
  • 13
  • 58
  • 123
einpoklum
  • 118,144
  • 57
  • 340
  • 684

1 Answers1

2

In MSVC this is called "incremental linking". Interestingly, what I've found that GCC might support that to some extent, try using the "-Wl,-i" or "-Wl,-r" parameters to GCC (should be actually also supported by CLang, as these "-Wl" parameters are just passed to ld).

I never used it before, but I made this work with the following makefile:

OBJS := a.o b.o c.o main.o

all:    test_app

test_app:   test_app.reloc
    g++ -o $@ $^

# build a "relocatable" object for incremental linking (either -i or -r)
test_app.reloc: $(OBJS)
    g++ -Wl,-i -nostdlib -nostartfiles -o $@ $^

$(OBJS):    makefile

%.o:    %.cpp
    g++ -c -o $@ $<

This builds the app, but I'm not entirely sure what it does internally, if it really does something like "incremental linking" done in MSVC.

In particular, the parameter "-nostdlib" is necessary when using the "-Wl,-i" so that the default libs will not be passed to the ld (which then can't find them - without it I had the error "/usr/bin/ld: cannot find -lgcc_s").

Another version which might actually work better (not sure, this would need to be tested on a bigger application to see if there is some gain in the link time for single object updates):

OBJS := a.ro b.ro c.ro main.ro

all:    test_app

test_app:   $(OBJS)
    g++ -o $@ $^

%.o:    %.cpp
    g++ -c -o $@ $<

%.ro:   %.o
    g++ -Wl,-i -nostdlib -nostartfiles -o $@ $<
  • Basically creating relocatable file for each object (which might be perhaps a significant portion of the linkage of obj files into the executable) and then just updating the relocatables necessary. For the final link step using the relocatables to link everything together (but part of the linkage has been already done before).

It is also possible to create "groups" of object files to be grouped in a single relocatable, so that there will be less of them at the end (not sure if that would bring anything at the end though).

EmDroid
  • 5,918
  • 18
  • 18
  • Can you write a few more words about what the workflow is like when I use these options? e.g. suppose my source files are `a.cpp`, `b.cpp` and `c.cpp`, compiled separately, with no includes, and I want to be able to do less linking work when I recompile `a.cpp`. What should I run initially and what should I run with the new `a.o`? – einpoklum Feb 15 '17 at 12:51
  • Update with an example makefile – EmDroid Feb 15 '17 at 13:36
  • About the second makefile: The point is that gcc would be able to easily combine .ro files into a single executable. Ok, but then - why do we even need the .cpp -> .o step at all? Why not just do .cpp -> .ro immediately? – einpoklum Feb 15 '17 at 14:19
  • Yes, I think that should work the same way as well, by creating the relocatable obj directly. – EmDroid Feb 15 '17 at 15:18
  • `ld -r` does relocatable link which is quite different from incremental link. – yugr Feb 15 '17 at 15:48
  • Yes, the point is that you only need to recompile some of the relocatable objects and reuse the other (which did not change), so that might result in faster linking afterwards (I guess the second makefile would be more appropriate). But as said, I actually have no extensive experience with this, it is just what I've found. Interestingly though, the `ld -i` is really called "incremental link" in the ld man and apparently it is a synonym for `ld -r`. – EmDroid Feb 15 '17 at 15:58
  • @axalis: I only ever only recompile the files that change. The question is whether linking .ro files is much faster than linking .r files (I would think not actually - unless you re-use the binary in the linking). – einpoklum Feb 15 '17 at 16:22