Recursive search in VPATH?

Question

My C++ project has source files organized in nested subdirectories of ./src. I have a pattern rule in my makefile which compiles all of the .cpp source files into objects:

$(OBJDIR)/%.o: %.cpp makefile
    $(CXX) -c $< -o $@

Since I am using this pattern rather than writing a compilation rule for each source file, I need to tell make to look recursively through ./src for these prerequisites. Right now I have:

VPATH := $./src/:./src/folder1:./src/folder2:./src/folder3

This works, but it feels pretty inelegant and also causes bugs when I inevitably forget to add in a new folder.

Hoping someone has a better solution!

Does this help: [Makefile rule that depends on all files under a directory (including within subdirectories)](https://stackoverflow.com/questions/14289513/makefile-rule-that-depends-on-all-files-under-a-directory-including-within-subd) — Reinier Torenbeek, Jan 15 '19 at 03:18
I couldn't get this to work with `wildcard`, because it's meant to match on filenames in a directory, not recursive subdirectories. — rampatowl, Jan 15 '19 at 03:20
Better to not use VPATH at all, and actually specify the full paths. I'm honestly not sure how you would be getting the truncated paths in the first place. — o11c, Jan 15 '19 at 03:20
@o11c Can you elaborate on this? For context, I got started down this path (generic pattern for compiling object files) as a means of doing automatic dependency management, and it seems to work well. I'm basically following this answer: https://stackoverflow.com/questions/54162861/how-to-avoid-forgetting-dependencies-in-make-cmake?noredirect=1#comment95160092_54162861 — rampatowl, Jan 15 '19 at 03:23
Normally automated dependency management looks something like: `main-objects := obj/main.o obj/test-main.o; lib-objects := $(filter-out $(main-objects),$(patsubst src/%.c,obj/%.o,$(shell find src -name '*.c')))`. Vpath is not useful at any stage. — o11c, Jan 15 '19 at 03:39
@o11c Pardon me if I'm being dense, I am new to this stuff. It seems that at some point you need a rule like `%.o: %.cpp`. But your `cpp` files are in a nested directory structure. Isn't the natural way of allowing them to be found as prereqs to edit VPATH? Is there a downside I am missing? — rampatowl, Jan 15 '19 at 04:07
Just use a `obj/%.o: src/%.cpp` rule, and `obj` will follow a mirrored nested structure. I'm working on collecting all my random Makefile knowledge into one easily-read repo ... — o11c, Jan 15 '19 at 04:57
Okay, I threw together a demo project for how I do it: https://github.com/o11c/makefile-demo - it's a bit hairier because it's generic, usually I hard-code some stuff. — o11c, Jan 15 '19 at 09:32
Ah, I see now. Thanks for the example, this is very helpful! — rampatowl, Jan 15 '19 at 15:28
Agree that it is a bad idea to use VPATH in `makefile` at all. I suggest you to use https://github.com/igagis/prorab/blob/master/wiki/TutorialBasicConcepts.md#adding-all-source-files-from-all-subdirectories-to-the-build — igagis, Jan 15 '19 at 22:41

Reinier Torenbeek · Accepted Answer · 2019-01-15T03:55:45.800

You can automate the building of the VPATH variable like yours by searching for subdirectories and replacing spaces with colons:

space :=
space +=
VPATH := $(subst $(space),:,$(shell find src -type d))

This assumes that you have no spaces in your directories or filenames.

With this approach, it is not clear to me what you would do if two source files in two different subdirectories have the same name -- but that seems to be more related to your overall setup than to your question about the VPATH specifically.

For the $(space) variable trick, see the nifty Escaping comma and space in GNU Make blog post.

score 1 · Answer 2 · answered Jan 15 '19 at 03:43

gmake itself does not have any functions for recursive directory traversal, so you have to resort to $(shell ...):

VPATH := $(shell find src -type d -print | tr '\012' ':' | sed 's/:$$//')

Tweak the shell script to get the right semantics. You want to use the := operator, in order to evaluate this one time.

Recursive search in VPATH?

2 Answers2