Relink Makefile target when list of dependencies changes

Question

A common problem I've seen with Makefiles is that executables (and libraries, for that matter) aren't necessarily relinked when the list of dependencies changes. For example:

SRCS=$(wildcard *.c)
CPPFLAGS=-MMD -MP
target: $(SRCS:.c=.o)
        $(CC) $(LDFLAGS) $^ -o $@
-include $(SRCS:.c=.d)

So far, so good: it automatically handles any changes to source files and their included dependencies. If a file is added (because of the wildcard, that means just adding it to the directory, but an explicit list would have the same problem), make sees it and relinks the target. However, when a source file is removed, make doesn't know that the target needs to be rebuilt.

How do I get make (gmake, specifically) to handle this situation?

Admittedly, it's not exactly a common situation, as removing a file will most likely mean that other files must be changed too, which would then force a relink anyway, but I've seen it happen before.

I've come up with two solutions, neither of which is ideal. First, if we make the source list explicit instead of wildcarded, we can make the target depend on the Makefile. Changing the Makefile to remove a source file then causes the target to be relinked. This has two problems. First, you have to strip out Makefile from $^ before linking, which is a bit ugly (but do-able with filter-out). Second, this Makefile is intended to be a template, included by other Makefiles (which specify the sources and the target) -- we have to make the target depend on that Makefile instead. Ugh.

The second solution is to include some logic like the following:

SRCS=$(wildcard *.c)
CPPFLAGS=-MMD -MP
target: $(SRCS:.c=.o)
        $(CC) $(LDFLAGS) $^ -o $@
        @echo '$$(if $$(filter-out $$(SRCS:.c=.o), $(SRCS:.c=.o)), .PHONY:target)' > target.d
-include $(SRCS:.c=.d)
-include target.d

This makes a target.d file that tracks the list of dependencies and forces the target to be rebuilt if anything is removed. It works, but it's ugly. It also can't account for any additional non-source dependencies specified by an including Makefile.

Is there an official way to do this, or at least a better way?

(As an aside, I'd also like to clean up the associated object and dependency files when the source file is removed, if possible. That might be possible by extending my second solution and making it even uglier.)

I'd guess that listing dependencies explicitly and depending on Makefile or `$(MAKEFILE_LIST)` (which also lists any including makefiles) is the most common solution. As a third alternative, you could also apply this trick to the SRCS macro: http://stackoverflow.com/questions/3236145/force-gnu-make-to-rebuild-objects-affected-by-compiler-definition/3237349#3237349 — slowdog, Jul 26 '11 at 22:12
Could you elaborate on "I've seen it happen before"? If a source file is removed, what's the point of rebuilding? Either the rebuild will fail, or that source was dead code. The only purposes I can think of are 1) slimming the executable by removing the dead binary code, 2) detecting missing files, and 3) detecting useless sources by destructive testing. None seems worth this level of effort to automate. — Beta, Jul 27 '11 at 04:36
Here's the primary motivation: the source files (C++, actually) only had static initializers in them, adding themselves to a global list of instantiations (not the best design, but it's what I was working with). Deleting a file didn't cause a relink (because none of the other files changed), and made for some really confusing debugging. It's a situation that can happen, but it's pretty rare. — Doug, Jul 31 '11 at 03:47

score 4 · Answer 1 · answered Nov 17 '11 at 13:40

To do it properly, you'll probably have to resort to Paul Smith's Advanced Auto-Dependency Generation. Paul is the current maintainer of GNU make, and has described all the common problems (and their solutions!) concerning dependency generation.

The other white-papers on his website are recommended as well.

I use his tips and tricks for several years now, and - even if it gets complicated a little occasionally - studying them has been worth every second.

score 0 · Answer 2 · answered Jul 29 '11 at 01:24

You need a smarter version of gmake. ElectricMake is a gmake-compatible replacement that includes a ledger feature that basically makes the up-to-date check more sophisticated than just checking that the output is newer than the inputs. In particular, the ledger will hash the list of inputs for the target and if that list changes, it will cause the output to be rebuilt:

# cat Makefile
out: a b c
        @echo out from $^
        @touch out
# emake --emake-ledger=timestamp
out from a b c
# emake --emake-ledger=timestamp
make: `out' is up to date.
# vi Makefile ;# Remove 'c' from the prereq list.
# cat Makefile
out: a b
        @echo out from $^
        @touch out
# emake --emake-ledger=timestamp
out from a b

If you want to give it a try, you can get an eval copy.

I'll have to look into emake -- it has some the features that sound promising. But for now I'm extremely limited on what tools I'm allowed to use in our development environment. — Doug, Jul 31 '11 at 03:52

score 0 · Answer 3 · answered Jul 29 '11 at 13:40

Ideally, the should be only one make pattern rule to do all the linking (okay, one for executables, one for shared libraries and one for archives). Something like:

# rules.mk
$(BUILD_DIR}/exe/% : rules.mk
    g++ -o $@ $(filter-out %.mk,$^)

Next, when you define build targets make them depend on its own makefile:

# project_a.mk
$(BUILD_DIR}/exe/a : project_a.mk ${a_obj}

Normally, this dependency would be wrapped in a macro:

define EXE_TARGET
  $(BUILD_DIR}/exe/${1} : ${2}
  $(BUILD_DIR}/exe/${1} : $(lastword $(MAKEFILE_LIST))
endef

so in project_a.mk it'd do:

# project_a.mk
$(eval $(call EXE_TARGET,a,${a_obj}))

Relink Makefile target when list of dependencies changes

3 Answers3

Linked