Makefiles with source files in different directories

Question

I have a project where the directory structure is like this:

                         $projectroot
                              |
              +---------------+----------------+
              |               |                |
            part1/          part2/           part3/
              |               |                |
       +------+-----+     +---+----+       +---+-----+
       |      |     |     |        |       |         |
     data/   src/  inc/  src/     inc/   src/       inc/

How should I write a makefile that would be in part/src (or wherever really) that could comple/link on the c/c++ source files in part?/src ?

Can I do something like -I$projectroot/part1/src -I$projectroot/part1/inc -I$projectroot/part2/src ...

If that would work, is there an easier way to do it. I've seen projects where there is a makefile in each of the corresponding part? folders. [in this post I used the question mark like in bash syntax]

http://stackoverflow.com/questions/7123431/building-multiple-executables-with-similar-rules/7321954#7321954 — Maxim Egorushkin, Sep 30 '12 at 11:54
In the original gnu manual ( https://www.gnu.org/software/make/manual/html_node/Phony-Targets.html ) under Phony Targets there is a sample on `recursive invocation`, that coule be quite elegant. — Frank N, Sep 22 '14 at 08:03

score 134 · Accepted Answer · edited Oct 26 '16 at 21:24

134

The traditional way is to have a Makefile in each of the subdirectories (part1, part2, etc.) allowing you to build them independently. Further, have a Makefile in the root directory of the project which builds everything. The "root" Makefile would look something like the following:

all:
    +$(MAKE) -C part1
    +$(MAKE) -C part2
    +$(MAKE) -C part3

Since each line in a make target is run in its own shell, there is no need to worry about traversing back up the directory tree or to other directories.

I suggest taking a look at the GNU make manual section 5.7; it is very helpful.

edited Oct 26 '16 at 21:24

akhan

2,952
2
22
13

answered Jul 16 '09 at 18:10

27

This should be `+$(MAKE) -C part1` etc. This allows Make's job control to work into the subdirectories. – ephemient Jul 16 '09 at 19:11
29

This is a classic approach and is widely used, but it is sub-optimal in several ways that get worse as the project grow. Dave Hinton has the pointer to follow. – dmckee --- ex-moderator kitten Jul 16 '09 at 19:35
5

What does the + symbol mean? – Izzo Sep 20 '22 at 02:56

score 98 · Answer 2 · edited May 19 '21 at 13:06

98

If you have code in one subdirectory dependent on code in another subdirectory, you are probably better off with a single makefile at top-level.

See Recursive Make Considered Harmful for the full rationale, but basically you want make to have the full information it needs to decide whether or not a file needs to be rebuilt, and it won't have that if you only tell it about a third of your project.

The link above seems to be not reachable. The same document is reachable here:

edited May 19 '21 at 13:06

General Grievance

4,555
31
31
45

answered Jul 16 '09 at 18:29

dave4420

46,404
6
118
152

4

Thank you, was not aware of this. It's very useful to know the "right way" of doing things instead of ways that "just work" or are accepted as standard. – tjklemz Mar 22 '13 at 18:55
4

Recursive make was considered harmful, back when it really didn't work well. It isn't considered harmful these days, in fact, it's how the autotools / automake manage larger projects. – Edwin Buck Feb 03 '18 at 05:17
1

As of September 2020, all of these links seem unreachable. You can find it googling though – mizkichan Sep 19 '20 at 11:44

score 44 · Answer 3 · answered Jul 16 '09 at 18:46

44

The VPATH option might come in handy, which tells make what directories to look in for source code. You'd still need a -I option for each include path, though. An example:

CXXFLAGS=-Ipart1/inc -Ipart2/inc -Ipart3/inc
VPATH=part1/src:part2/src:part3/src

OutputExecutable: part1api.o part2api.o part3api.o

This will automatically find the matching partXapi.cpp files in any of the VPATH specified directories and compile them. However, this is more useful when your src directory is broken into subdirectories. For what you describe, as others have said, you are probably better off with a makefile for each part, especially if each part can stand alone.

answered Jul 16 '09 at 18:46

CodeGoat

1,186
8
6

4

I cant believe this simple perfect answer hasn't got more votes. Its a +1 from me. – Nicholas Hamilton Jun 15 '14 at 17:00
3

I had some common source files in a higher directory for several disparate projects in subfolders, `VPATH=..` worked for me! – EkriirkE Dec 02 '16 at 23:05
Perfect, so simple. thanks! – Sujay Phadke Jul 10 '23 at 09:48

RC. · Answer 4 · 2009-07-16T19:23:01.763

You can add rules to your root Makefile in order to compile the necessary cpp files in other directories. The Makefile example below should be a good start in getting you to where you want to be.

CC=g++
TARGET=cppTest
OTHERDIR=../../someotherpath/in/project/src

SOURCE = cppTest.cpp
SOURCE = $(OTHERDIR)/file.cpp

## End sources definition
INCLUDE = -I./ $(AN_INCLUDE_DIR)  
INCLUDE = -I.$(OTHERDIR)/../inc
## end more includes

VPATH=$(OTHERDIR)
OBJ=$(join $(addsuffix ../obj/, $(dir $(SOURCE))), $(notdir $(SOURCE:.cpp=.o))) 

## Fix dependency destination to be ../.dep relative to the src dir
DEPENDS=$(join $(addsuffix ../.dep/, $(dir $(SOURCE))), $(notdir $(SOURCE:.cpp=.d)))

## Default rule executed
all: $(TARGET)
        @true

## Clean Rule
clean:
        @-rm -f $(TARGET) $(OBJ) $(DEPENDS)


## Rule for making the actual target
$(TARGET): $(OBJ)
        @echo "============="
        @echo "Linking the target $@"
        @echo "============="
        @$(CC) $(CFLAGS) -o $@ $^ $(LIBS)
        @echo -- Link finished --

## Generic compilation rule
%.o : %.cpp
        @mkdir -p $(dir $@)
        @echo "============="
        @echo "Compiling $<"
        @$(CC) $(CFLAGS) -c $< -o $@


## Rules for object files from cpp files
## Object file for each file is put in obj directory
## one level up from the actual source directory.
../obj/%.o : %.cpp
        @mkdir -p $(dir $@)
        @echo "============="
        @echo "Compiling $<"
        @$(CC) $(CFLAGS) -c $< -o $@

# Rule for "other directory"  You will need one per "other" dir
$(OTHERDIR)/../obj/%.o : %.cpp
        @mkdir -p $(dir $@)
        @echo "============="
        @echo "Compiling $<"
        @$(CC) $(CFLAGS) -c $< -o $@

## Make dependancy rules
../.dep/%.d: %.cpp
        @mkdir -p $(dir $@)
        @echo "============="
        @echo Building dependencies file for $*.o
        @$(SHELL) -ec '$(CC) -M $(CFLAGS) $< | sed "s^$*.o^../obj/$*.o^" > $@'

## Dependency rule for "other" directory
$(OTHERDIR)/../.dep/%.d: %.cpp
        @mkdir -p $(dir $@)
        @echo "============="
        @echo Building dependencies file for $*.o
        @$(SHELL) -ec '$(CC) -M $(CFLAGS) $< | sed "s^$*.o^$(OTHERDIR)/../obj/$*.o^" > $@'

## Include the dependency files
-include $(DEPENDS)

I know this is pretty old by now but, aren't both SOURCE and INCLUDE overwritten by another assignment one line later? — skelliam, Oct 16 '15 at 16:15
This approach violates the DRY principle. It is a bad idea to duplicate the code for each "other directory" — Jesus H, Apr 27 '17 at 19:08

score 26 · Answer 5 · answered May 16 '12 at 04:32

If the sources are spread in many folders, and it makes sense to have individual Makefiles then as suggested before, recursive make is a good approach, but for smaller projects I find it easier to list all the source files in the Makefile with their relative path to the Makefile like this:

# common sources
COMMON_SRC := ./main.cpp \
              ../src1/somefile.cpp \
              ../src1/somefile2.cpp \
              ../src2/somefile3.cpp \

I can then set VPATH this way:

VPATH := ../src1:../src2

Then I build the objects:

COMMON_OBJS := $(patsubst %.cpp, $(ObjDir)/%$(ARCH)$(DEBUG).o, $(notdir $(COMMON_SRC)))

Now the rule is simple:

# the "common" object files
$(ObjDir)/%$(ARCH)$(DEBUG).o : %.cpp Makefile
    @echo creating $@ ...
    $(CXX) $(CFLAGS) $(EXTRA_CFLAGS) -c -o $@ $<

And building the output is even easier:

# This will make the cbsdk shared library
$(BinDir)/$(OUTPUTBIN): $(COMMON_OBJS)
    @echo building output ...
    $(CXX) -o $(BinDir)/$(OUTPUTBIN) $(COMMON_OBJS) $(LFLAGS)

One can even make the VPATH generation automated by:

VPATH := $(dir $(COMMON_SRC))

Or using the fact that sort removes duplicates (although it should not matter):

VPATH := $(sort  $(dir $(COMMON_SRC)))

this works great for smaller projects where you just want to add some custom libraries and have them in a separate folder. Thank you very much — zitroneneis, Aug 08 '12 at 11:24

IlDan · Answer 6 · 2012-09-30T11:42:19.613

6

I think it's better to point out that using Make (recursive or not) is something that usually you may want to avoid, because compared to today tools, it's difficult to learn, maintain and scale.

It's a wonderful tool but it's direct use should be considered obsolete in 2010+.

Unless, of course, you're working in a special environment i.e. with a legacy project etc.

Use an IDE, CMake or, if you're hard cored, the Autotools.

(edited due to downvotes, ty Honza for pointing out)

edited Sep 30 '12 at 11:42

answered Jul 16 '09 at 18:34

IlDan

6,851
3
33
34

1

It'd be nice to have the downvotes explained. I used to make my Makefiles myself too. – IlDan Jul 16 '09 at 18:48
3

I'm upvoting for the "don't do that" suggestion -- KDevelop has a very graphical interface to configure Make and other build systems, and while I do write Makefiles myself, KDevelop is what I give to my workmates -- but I don't think the Autotools link helps. To understand Autotools, you need to understand m4, libtool, automake, autoconf, shell, make, and basically the entire stack. – ephemient Jul 16 '09 at 20:36
8

The downvotes are probably because you're answering your own question. The question wasn't about whether or not one should use Makefiles. It was about how to write them. – Honza Sep 29 '12 at 11:57
ty Honza, that makes sense. Reworded – IlDan Sep 30 '12 at 11:42
8

it would be nice if you could, it in a few sentences, explain how the modern tools make life easier that writing a Makefile. Do they provide a higher level abstraction? or what? – Abhishek Anand Apr 14 '14 at 19:46
1

xterm, vi, bash, makefile== rule the world, cmake is for people who can't hack code. – μολὼν.λαβέ Jul 21 '16 at 00:53
Your recommendation is very myopic. – Vroomfondel May 19 '21 at 13:33

score 5 · Answer 7 · answered Apr 23 '19 at 03:35

I was looking for something like this and after some tries and falls i create my own makefile, I know that's not the "idiomatic way" but it's a begining to understand make and this works for me, maybe you could try in your project.

PROJ_NAME=mono

CPP_FILES=$(shell find . -name "*.cpp")

S_OBJ=$(patsubst %.cpp, %.o, $(CPP_FILES))

CXXFLAGS=-c \
         -g \
        -Wall

all: $(PROJ_NAME)
    @echo Running application
    @echo
    @./$(PROJ_NAME)

$(PROJ_NAME): $(S_OBJ)
    @echo Linking objects...
    @g++ -o $@ $^

%.o: %.cpp %.h
    @echo Compiling and generating object $@ ...
    @g++ $< $(CXXFLAGS) -o $@

main.o: main.cpp
    @echo Compiling and generating object $@ ...
    @g++ $< $(CXXFLAGS)

clean:
    @echo Removing secondary things
    @rm -r -f objects $(S_OBJ) $(PROJ_NAME)
    @echo Done!

I know that's simple and for some people my flags are wrong, but as i said this is my first Makefile to compile my project in multiple dirs and link all of then together to create my bin.

I'm accepting sugestions :D

score 3 · Answer 8 · answered Feb 15 '12 at 19:15

RC's post was SUPER useful. I never thought about using the $(dir $@) function, but it did exactly what I needed it to do.

In parentDir, have a bunch of directories with source files in them: dirA, dirB, dirC. Various files depend on the object files in other directories, so I wanted to be able to make one file from within one directory, and have it make that dependency by calling the makefile associated with that dependency.

Essentially, I made one Makefile in parentDir that had (among many other things) a generic rule similar to RC's:

%.o : %.cpp
        @mkdir -p $(dir $@)
        @echo "============="
        @echo "Compiling $<"
        @$(CC) $(CFLAGS) -c $< -o $@

Each subdirectory included this upper-level makefile in order to inherit this generic rule. In each subdirectory's Makefile, I wrote a custom rule for each file so that I could keep track of everything that each individual file depended on.

Whenever I needed to make a file, I used (essentially) this rule to recursively make any/all dependencies. Perfect!

NOTE: there's a utility called "makepp" that seems to do this very task even more intuitively, but for the sake of portability and not depending on another tool, I chose to do it this way.

Hope this helps!

score 1 · Answer 9 · answered Apr 05 '12 at 00:16

1

Recursive Use of Make

all:
    +$(MAKE) -C part1
    +$(MAKE) -C part2
    +$(MAKE) -C part3

This allows for make to split into jobs and use multiple cores

answered Apr 05 '12 at 00:16

EhevuTov

20,205
16
66
71

2

How is this better than doing something like `make -j4`? – devin May 16 '12 at 16:26
1

@devin as I see it, it's not *better*, it just *enables* `make` to use job control at all. When running separate process of make, it is not subject to job control. – Michael Pankov Mar 01 '13 at 05:32

score -1 · Answer 10 · edited Sep 10 '13 at 07:00

-1

I suggest to use autotools:

//## Place generated object files (.o) into the same directory as their source files, in order to avoid collisions when non-recursive make is used.

AUTOMAKE_OPTIONS = subdir-objects

just including it in Makefile.am with the other quite simple stuff.

Here is the tutorial.

edited Sep 10 '13 at 07:00

Baby Groot

4,637
39
52
71

answered Sep 10 '13 at 06:35

user2763812

9

Makefiles with source files in different directories

10 Answers10

Linked

Related