Evolving a Makefile From Flat Directory Structure to Sub-Directory Structure

Question

SEE UPDATES BELOW

Research Done: I'm finding learning how to evolve Makefiles from one situation to another is difficult. There are a ton of questions and answers out there but few of them actually show how a Makefile can evolve as your project changes. They also all seem to use various different techniques and idioms of Makefiles so translating between one question and another can be tricky when you are learning Makefiles for the first time, as I am.

Problem: My problem is that I have a project that started at as a flat directory structure but then is migrating to a structure with sub-directories. What I can't do is get my Makefile to along for the ride.

First I'll show what I created that works and then I show how I want it to evolve and how that doesn't work.

Flat Directory Structure, Working Makefile

I have project directory that has all my C files and one header file plus my Makefile:

project
  Makefile
  c8_asm.c
  c8_dasm.c
  c8_terp.c
  chip8.h

Here is my Makefile (which works just fine):

CC = gcc

CFLAGS += -c -Wall -std=c99
CFLAGS += -D_POSIX_C_SOURCE=200809L

LDLIBS += -lm

# Targets

all: c8_dasm c8_asm c8_terp

c8_dasm: c8_dasm.o
    $(CC) $(LDLIBS) c8_dasm.o -o $@

c8_asm: c8_asm.o
    $(CC) $(LDLIBS) c8_asm.o -o $@

c8_terp: c8_terp.o
    $(CC) $(LDLIBS) c8_terp.o -o $@

# Using implicit rules for updating an '.o' file from a correspondingly
# named '.c' file.

c8_dasm.o: chip8.h
c8_asm.o: chip8.h
c8_terp.o: chip8.h

.PHONY: clean
clean:
    rm c8_dasm c8_asm c8_terp c8_dasm.o c8_asm.o c8_terp.o

I get all my .o files and my executables are created in the project directory.

Evolving The Project

But what I wanted to do is have my sources files (all .c and .h) in a src directory. I wanted to build into an obj directory and have the executables go in a bin directory. So my project would look like this:

project
  src
    c8_asm.c
    c8_dasm.c
    c8_terp.c
    chip8.h
  Makefile

Sub-Directory Structure, Makefile NOT Working

To accommodate the above, I changed my Makefile accordingly:

CC = gcc

CFLAGS += -c -Wall -std=c99
CFLAGS += -D_POSIX_C_SOURCE=200809L

LDLIBS += -lm

SRC_DIR = src
OBJ_DIR = obj
BIN_DIR = bin

SOURCES := $(wildcard $(SRC_DIR)/*.c)
OBJECTS := $(SOURCES:$(SRC_DIR)/%.c=$(OBJ_DIR)/%.o)

MKDIR_P ?= mkdir -p

# Targets

all: $(BIN_DIR)/c8_dasm $(BIN_DIR)/c8_asm $(BIN_DIR)/c8_terp

$(BIN_DIR)/c8_dasm: $(OBJ_DIR)/c8_dasm.o
    $(CC) $(LDLIBS) $(OBJ_DIR)/c8_dasm.o -o $@

$(BIN_DIR)/c8_asm: $(OBJ_DIR)/c8_asm.o
    $(CC) $(LDLIBS) $(OBJ_DIR)/c8_asm.o -o $@

$(BIN_DIR)/c8_terp: $(OBJ_DIR)/c8_terp.o
    $(MKDIR_P) $(dir $@)
    $(CC) $(LDLIBS) $(OBJ_DIR)/c8_terp.o -o $@

$(OBJECTS): $(OBJ_DIR)/%.o : $(SRC_DIR)/%.c
    $(MKDIR_P) $(dir $@)
    $(CC) $< -o $(OBJ_DIR)/$@

# Using implicit rules for updating an '.o' file from a correspondingly
# named '.c' file.

$(OBJ_DIR)/c8_dasm.o: $(SRC_DIR)/chip8.h
$(OBJ_DIR)/c8_asm.o: $(SRC_DIR)/chip8.h
$(OBJ_DIR)/c8_terp.o: $(SRC_DIR)/chip8.h

.PHONY: clean
clean:
    rm -r $(BUILD_DIR)
    rm $(OBJECTS)

Upon running this I get the following:

mkdir -p obj/obj/
gcc src/c8_dasm.c -o obj/c8_dasm.o
gcc -lm obj/c8_dasm.o -o bin/c8_dasm
ld: can't link with a main executable file 'obj/c8_dasm.o' for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [bin/c8_dasm] Error 1

I wanted to stop here and get some assistance because I fear I'm making this Makefile for complicated than it need be and I'm trying to avoid getting into bad habits.

I'm hoping to hear opinions about what I'm not conceptualizing correctly here.

FIRST UPDATE

I managed to take it bit by bit and get it mostly working. Here is what I ended up with:

CC = gcc

CFLAGS += -c -Wall -std=c99
CFLAGS += -D_POSIX_C_SOURCE=200809L

LDLIBS += -lm

# Directories.

SRC_DIR = src
BIN_DIR = bin

$(shell mkdir -p $(BIN_DIR))

# Patterns for files.

SOURCES := $(wildcard $(SRC_DIR)/*.c)
OBJECTS := $(SOURCES:$(SRC_DIR)/%.c=$(SRC_DIR)/%.o)

EXECUTABLES := c8_dasm c8_asm c8_terp

# Targets

all: $(EXECUTABLES)

c8_dasm: $(SRC_DIR)/c8_dasm.o
    $(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$@
    @echo "C8 Disassembler Built"

c8_asm: $(SRC_DIR)/c8_asm.o
    $(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$@
    @echo "C8 Assembler Built"

c8_terp: $(SRC_DIR)/c8_terp.o
    $(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$@
    @echo "C8 Interpreter Built"

# Using implicit rules for updating an '.o' file from a correspondingly
# named '.c' file.

c8_dasm.o: $(SRC_DIR)/chip8.h
c8_asm.o: $(SRC_DIR)/chip8.h
c8_terp.o: $(SRC_DIR)/chip8.h

.PHONY: clean
clean:
    rm $(OBJECTS)
    rm -r $(BIN_DIR)

Of course, as I'm finding with Make this leads to other obscure problems. For example doing this:

make
make clean

works fine. Meaning all files are generated and the files are cleaned, including the bin directory.

However, if I do this:

make c8_dasm
make clean

This builds fine. But the clean fails to delete the bin directory (although it does delete the object files). This happens regardless of what individual executable I try to build.

No amount of searching is helping me find out why that is.

SECOND UPDATE

I found that problem was solved as well. It just required using the "-f" for the rm statements in the clean target.

THIRD UPDATE

To get the object file directory part working, I tried (from this: path include and src directory makefile) to construct my Makefile as follows:

CC = gcc

CFLAGS += -c -Wall -std=c99
CFLAGS += -D_POSIX_C_SOURCE=200809L

LDLIBS += -lm

SRC_DIR = src
OBJ_DIR = obj
BIN_DIR = bin

$(shell mkdir -p $(BIN_DIR))
$(shell mkdir -p $(OBJ_DIR))

SOURCES := $(wildcard $(SRC_DIR)/*.c)
OBJECTS := $(SOURCES:$(SRC_DIR)/%.c=$(OBJ_DIR)/%.o)

EXECUTABLES := c8_dasm c8_asm c8_terp

all: $(EXECUTABLES)

c8_dasm: $(SRC_DIR)/c8_dasm.o
    $(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$@
    @echo "C8 Disassembler Built"

c8_asm: $(SRC_DIR)/c8_asm.o
    $(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$@
    @echo "C8 Assembler Built"

c8_terp: $(SRC_DIR)/c8_terp.o
    $(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$@
    @echo "C8 Interpreter Built"

$(OBJ_DIR)/%.o: $(SRC_DIR)/%.c
    $(CC) $(CFLAGS) -c $< -o $(BIN_DIR)/$@

.PHONY: clean
clean:
    rm -rf $(BIN_DIR)
    rm -f $(OBJECTS)

I was able to condense the original three lines using chip8.h into one target but I have no way to know if that's correct. It compiles at least. I also changed the OBJECTS line to reflect the new OBJ_DIR I created.

However, this doesn't put the object files in the right place. It still puts them in the src directory rather than the obj directory.

score 1 · Answer 1 · answered Feb 14 '19 at 17:50

1

This is why it makes sense to not do anything complicated with Makefiles. Just put the actual directory names in your commands. Never rely on wildcards.

People using C and C++ and using Makefiles spend too much time trying to get those to work rather than just actually getting things done. That's why you see so many of the questions that you see and why the answers vary so much.

In your specific case, your targets don't always have to contain the directory and that's part of the problem. The rules getting generated don't have an actual target in your file because of the directories you are prepending to everything. You have to think in terms of what is getting generated by each target: meaning, the output. So if c8_dasm is getting output, that's your target. The directory has nothing to do with that. So you need to remove all of your directory substitutions where they aren't needed.

But before doing that, ask yourself this: if your first solution was working, why change it? It's better to not even do directories when you're using Make. Just have everything in the same directory as you started off with. You can even see that this allows your Makefile to be much cleaner.

answered Feb 14 '19 at 17:50

Micah Condritch

59
5

Yeah, I started adding a lot of those directory paths in the attempt to get something -- anything! -- to work! :) The documentation on this is actually fairly good but I find most of it was written without the idea of making it cohesive. That being said, I have seen a lot of use of wildcards so I'm not sure why you are suggesting to never use them. I'm also not sure why a project with directories would be so hard for Make. I know many people do this so this is clearly just my lack of understanding of how. – Jeff Nyman Feb 14 '19 at 19:07
1

See my update above. I got part of it working but it just leads to other oddities. – Jeff Nyman Feb 14 '19 at 19:11
The clean (from your update) should work fine either way. – Micah Condritch Feb 14 '19 at 19:22
Good job! You actually just showed yourself how to evolve a Makefile. It looks like the last thing you need to still add is to have your object files generated in the obj directory, assuming you still want to do that. – Micah Condritch Feb 14 '19 at 20:00
I doubt I'll do that at this point. I've tried it but it just leads into chaos, particularly with that need for my chip8.h file. If I do figure that out, I'll likely post that as the answer unless someone beats me to it. – Jeff Nyman Feb 14 '19 at 20:13
As a hint, you're going to have trouble with your lines that are on your executable targets. For example "c8_dasm: $(SRC_DIR)/c8_dasm.o". You're also going want to figure out a way to condense those lines that all reference "$(SRC_DIR)/chip8.h". Also now that you're doing the "$(BIN_DIR)/$@" in each of your executables, that's no longer going to work to generalize for your obj directory. (Because it would try to build in bin/obj/c8_dasm.o and so on.) – Micah Condritch Feb 14 '19 at 20:28
1

See my third update. One of the challenges is that not all of my object files are going to be needed for each executable. For example, eventually I'm going to have a `dasm.c` and an `asm.c` file and thus a `dasm.o` and an `asm.o` object file. But the `dasm.o` is only relevant to `c8_dasm` executable whereas `asm.o` will only be relevant to the `c8_asm` executable. I've seen examples of how to handle this where you just have on executable and all object files are needed for it. The examples for multiple executables where only some object files are needed for each are often extremely unclear. – Jeff Nyman Feb 14 '19 at 20:43

score 1 · Accepted Answer · answered Feb 14 '19 at 21:17

I believe I may have figured this out. Below is my Makefile. It seems to do what I want. It does the following:

Compiles all object files into the obj directory.
Compiles and links so that executables are generated in the bin directory.
Recognizes if any .c files are changed and recompiles accordingly.
Recognizes if the .h file is changed and recompiles all C files that reference it.

This seems to satisfy all the criteria but I can't tell if I've painted myself into some corner that I can't see yet.

CC = gcc

CFLAGS += -c -Wall -std=c99
CFLAGS += -D_POSIX_C_SOURCE=200809L

LDLIBS += -lm

SRC_DIR = src
OBJ_DIR = obj
BIN_DIR = bin

$(shell mkdir -p $(BIN_DIR))
$(shell mkdir -p $(OBJ_DIR))

SOURCES := $(wildcard $(SRC_DIR)/*.c)
OBJECTS := $(SOURCES:$(SRC_DIR)/%.c=$(OBJ_DIR)/%.o)

EXECUTABLES := c8_dasm c8_asm c8_terp

all: $(EXECUTABLES)

c8_dasm: $(OBJ_DIR)/c8_dasm.o
    $(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$@
    @echo "C8 Disassembler Built"

c8_asm: $(OBJ_DIR)/c8_asm.o
    $(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$@
    @echo "C8 Assembler Built"

c8_terp: $(OBJ_DIR)/c8_terp.o
   $(CC) $^ $(LDLIBS) -o $(BIN_DIR)/$@
   @echo "C8 Interpreter Built"

$(OBJ_DIR)/%.o: $(SRC_DIR)/%.c $(SRC_DIR)/chip8.h
    $(CC) $(CFLAGS) -c $< -o $@

.PHONY: clean
clean:
    rm -rf $(BIN_DIR)
    rm -rf $(OBJ_DIR)

score 0 · Answer 3 · answered Feb 14 '19 at 20:57

Stackoverflow is whining about too many comments, so I'll make this another "answer." After our back-and-forth to my original comment, your last comment is correct. That's what I wanted you to see.

Understand that you can't use Make to do what you want to do exactly.

So here's really the answer: You can't create multiple executables AND with only some of the object files applying to each one AND while using a directory structure. Make is in no way capable of handling that.

Right now you're trying to use Make in a way that it wasn't intended for which is why you're running into so many problems. If you keep playing around you're going to run into is a series of errors that say "duplicate symbol" because you will be compiling each of your files multiple times for each executable, assuming you follow most of the advice you'll find.

Check out this How can I create a Makefile for C projects with SRC, OBJ, and BIN subdirectories? to see what I mean. That one works because all object files are being used to create a single executable. But as you've stated, that's not going to be the case for you. And that's what Make can't handle. That's why you're not finding an answer to that.

And while your chip8.h file is now not going to cause problems in terms of allowing you to compile, your Makefile with that third update would not recognize when the chip8.h file itself has changed. You would have to change a .c file to force a recompile so that changes to your .h were recognized. So you either have to stick with your second update or use something other than Make.

Hmm. See my answer below. It _seems_ to do what I want, but based on what you're saying, I'm not sure if I'm deluding myself. — Jeff Nyman, Feb 14 '19 at 21:17
Awesome! I hadn't seen this and didn't follow up but it's cool that you figured out what you needed to do. — Micah Condritch, May 19 '19 at 23:20

Evolving a Makefile From Flat Directory Structure to Sub-Directory Structure

3 Answers3