depending on directories in make

Question

This is a followup to my earlier question: SO 4403861 because the suggested solutions broke the dependencies, making the makefile useless. I can't figure out why.

I am using gnu make 3.82 I have a rule that works if the obj directory has been created:

objdir:=../obj
$(objdir)/%.o: %.C
    $(COMPILE) -MM -MT$(objdir)/$(notdir $@) $< -o $(DEPDIR)/$(notdir $(basename $<).d )
    $(COMPILE) -o $(objdir)/$(notdir $@ ) -c $<

However, if the obj directory isn't there, make fails. I wanted make to automatically create ../obj on demand, so I added what I thought was very simple:

$(objdir)/%.o: %.C $(objdir)
    $(COMPILE) -MM -MT$(objdir)/$(notdir $@) $< -o $(DEPDIR)/$(notdir $(basename $<).d )
    $(COMPILE) -o $(objdir)/$(notdir $@ ) -c $<

$(objdir):
   if [ ! -d $(objdir) ] ; then mkdir $(objdir) ; fi

When I do so, make always forces the compile, every time. Why? The mkdir should not happen unless there is no directory? Why are dependencies destroyed by this simple change?

As I commented in response to your question/comment to my answer on 4403861: _"Your rule says that the object file depends on the object directory - which means that if the object directory has changed since the object file was last built (for example, because you compiled another file), then your file needs to be rebuilt. Your overall build target depends on the object directory existing; the individual object files do not need rebuilding just because the directory changed."_ — Jonathan Leffler, Dec 15 '10 at 19:45

score 39 · Accepted Answer · edited Feb 13 '21 at 14:22

39

You also can try with Order-only prerequisites.

There is a similar example, of your question, available.

 OBJDIR := objdir
 OBJS := $(addprefix $(OBJDIR)/,foo.o bar.o baz.o)
 
 $(OBJDIR)/%.o : %.c
         $(COMPILE.c) $(OUTPUT_OPTION) $<
 
 all: $(OBJS)
 
 $(OBJS): | $(OBJDIR)
 
 $(OBJDIR):
         mkdir $(OBJDIR)

edited Feb 13 '21 at 14:22

Neuron

5,141
5
38
59

answered Dec 19 '10 at 06:45

Bhavik

609
2
6
12

slowdog · Answer 2 · 2010-12-15T18:57:26.730

20

As others have said, the problem is that a directory is considered "changed" whenever directory members are added or removed, so make sees your output directory changing all the time, and reruns all the compilations which you have told it depend on the output directory.

As an alternative to the workarounds described by others, recent GNU make versions have support for "order-only prerequisites". The description of order-only prerequisites in the manual includes an example of how to use them for directory creation.

edited Dec 15 '10 at 18:57

answered Dec 14 '10 at 23:15

slowdog

6,076
2
27
30

2

I don' thnk this is quite true. A directory changes when any of its members is added, removed, or renamed. (A directory is a file containing a mapping of filenames to files.) – reinierpost Dec 15 '10 at 00:00
I explain this here: http://stackoverflow.com/questions/3477418/suppress-make-rule-error-output/3554442#3554442 – Jack Kelly Dec 15 '10 at 04:47
reinierpost: thanks, corrected the answer. – slowdog Dec 15 '10 at 18:58

score 4 · Answer 3 · answered Jan 06 '11 at 10:54

Some sort of sentinel file will solve this quite nicely. Allows you to write a single pattern rule to create all folders, does not have any rogue recompilation issues, and is -j safe.

OBJDIR := objdir/
OBJS := $(addprefix ${OBJDIR},foo.o bar.o baz.o)

all: ${OBJS}

${OBJDIR}%.o : %.c ${OBJDIR}.sentinel
     ${COMPILE.c} ${OUTPUT_OPTION} $<

# A single pattern rule will create all appropriate folders as required
.PRECIOUS: %/.sentinel # otherwise make (annoyingly) deletes it
%/.sentinel:
     mkdir -p ${@D}
     touch $@

score 2 · Answer 4 · answered Dec 14 '10 at 18:07

2

As answered already, you cannot put a directory with output files as a dependency, because it is updated every time.

One solution (stated above) is to put a mkdir -p before each build, like this:

objdir:=../obj
$(objdir)/%.o: %.C
    @mkdir -p $(objdir) $(DEPDIR) ...
    $(COMPILE) -MM -MT$(objdir)/$(notdir $@) $< -o $(DEPDIR)/$(notdir $(basename $<).d )
    $(COMPILE) -o $(objdir)/$(notdir $@ ) -c $<

But I don't love this solution, because it forces running an extra process lots and lots of times. However, knowing the reason for the error, there is another way:

exe: dirs $(OBJ)

In the primary target, the one building the executable, just insert a target dirs before the objects. Since it's hit in order:

dirs:
    @mkdir -p $(objdir) $(DESTDIR) ...

and this is only done once for the entire build, rather than once per file.

answered Dec 14 '10 at 18:07

Dov

8,000
8
46
75

2

This is a little known annoyance, but you can't use `mkdir -p` in a parallel make. The reason being that if folders /a/b/c and /a/b/cc are being created in parallel there is a race condition between when `mkdir -p` checks say for /a directory existence and then creates it. Another process (make job) can create the directory in between the check and the action, so that the first job fails with EEXIST when creating the directory. – Maxim Egorushkin Jan 05 '11 at 17:42
1

And dependencies on directories must be order-only, as they already mentioned. – Maxim Egorushkin Jan 05 '11 at 17:42
As has been said, this solution is broken under `-j`. You _can_ use `mkdir -p`, but you have to get the dependencies right. Basically, you have to make all the _objects_ depend on the dir. Also, to stop recompiles due to the time of the dir changing, use a sentinel file (and a pattern rule). See below (I can't type the solution in this box, grrrr.). – bobbogo Jan 06 '11 at 10:39
With what you posted, there is no guarantee that the dirs target will be processed *before* $(OBJ) or that they won't happen at the same time (so dirs would get created but $(OBJ) fail since it didn't exist). – iheanyi Apr 29 '14 at 21:45

depending on directories in make

4 Answers4

Linked