0

I redesigned most of the Makefile files for my dissertation project in order to correctly reflect the workflow (Creating make rules for dependencies across targets in project's sub-directories). However, in a particular sub-directory (prepare), make always rebuilds all targets, even when there are no changes in dependencies. What could be the reason for this unexpected behavior?

NOTE: sf.done is a real file (of type, which I call "flag files"), located in a different sub-directory and created/updated upon completion of data collection (import) - dependent step for the target transform.

prepare/Makefile:

IMPORT_DIR=../import

prepare: import \
         transform \
         cleanup \
         merge \
         sample

import: $(IMPORT_DIR)/sf.done
transform: transform.done
cleanup: cleanup.done
merge: merge.done
sample: sample.done

transform.done: transform.R import
    @$(RSCRIPT) $(R_OPTS) $<
    @touch $@

cleanup.done: cleanup.R transform
    @$(RSCRIPT) $(R_OPTS) $<
    @touch $@

merge.done: merge.R cleanup
    @$(RSCRIPT) $(R_OPTS) $<
    @touch $@

sample.done: sample.R merge
    @$(RSCRIPT) $(R_OPTS) $<
    @touch $@

.PHONY: import transform cleanup merge sample clean

# remove intermediate files
clean:
    rm -f tmp*.bz2 *.Rdata .Rout

UPDATE:

IMPORT_DIR = ../import
IMPORT_DONE = $(IMPORT_DIR)/sf.done

prepare: import \
         transform \
         cleanup \
         merge \
         sample

import: import.done
transform: transform.done
cleanup: cleanup.done
merge: merge.done
sample: sample.done

import.done: $(IMPORT_DONE)
    @cd $(IMPORT_DIR) && $(MAKE)

transform.done: transform.R import.done
    @$(RSCRIPT) $(R_OPTS) $<
    @touch $@

cleanup.done: cleanup.R transform.done
    @$(RSCRIPT) $(R_OPTS) $<
    @touch $@

merge.done: merge.R cleanup.done
    @$(RSCRIPT) $(R_OPTS) $<
    @touch $@

sample.done: sample.R merge.done
    @$(RSCRIPT) $(R_OPTS) $<
    @touch $@

.PHONY: import transform cleanup merge sample clean
Community
  • 1
  • 1
Aleksandr Blekh
  • 2,462
  • 4
  • 32
  • 64

1 Answers1

7

You've declared the targets import transform cleanup merge sample clean to be .PHONY. That means that make will always consider them to be out of date.

Then you declare the various .done targets to depend on those .PHONY targets. Since the phony targets are always considered out of date, those .done targets always need to be updated, and the recipes always fire.

MadScientist
  • 92,819
  • 9
  • 109
  • 136
  • Thank you for your fast answer! My understanding was that any targets that are not real files HAVE to be declared as `.PHONY`. If this is true, as well as true is your statement about `make` considering them always out of date, then, how would you recommend me to handle this situation? – Aleksandr Blekh May 30 '14 at 14:14
  • OK, I redesigned the Makefile (please see UPDATE in my question), but `make` is still unhappy: `*** No rule to make target `../import/sf.done', needed by `import.done'`. I don't understand why `make` refers to `../import/sf.done` as TARGET, while it's simply a file-based dependency. – Aleksandr Blekh May 30 '14 at 14:48
  • 1
    I'm not sure what you mean _simply a file-based dependency_. The way make works is that when it reads a rule it first recursively treats every prerequisite as if it were a target and tries to make that target, and its depencies, and so on, until it cannot create any more, then it unrolls the recursion and runs the recipes of those targets whose prerequisite are newer. For every prerequisite it must either already exist, or make must know how to build it (there must be a rule to build it) or else make will fail. – MadScientist May 30 '14 at 15:23
  • 2
    Re your first question: you should do what you did: have the targets depend on the "flag files" (I usually call them "sentinel files") not the phony targets. Phony targets are mostly useful for providing command line targets. – MadScientist May 30 '14 at 15:30
  • 1
    Everything in your makefile relies on `import.done` and that step requires the `../import/sf.done` file because you've listed that as dependency. If that file is not there, nothing can run because you haven't told this make file how to create that target. – MrFlick May 30 '14 at 19:37
  • @MrFlick & MadScientist: Thank you for your feedback! I thought that in the `import` rule if `prepare/Makefile` I have provided the recipe for building `../import/sf.done`. I realize that the recipe is not the most elegant one, but IMHO it should work. I would gladly prefer to use a reference to the corresponding target in `import/Makefile`, but I haven't found documentation about that type of references (`inter-Makefile target references`?). – Aleksandr Blekh May 31 '14 at 03:10
  • 1
    `make` won't just randomly start running recipes in the hopes that one of them will create the file! If the file doesn't exist, you have to tell make how to build it. You do that by declaring a rule where the file to be built is the _target_ of that rule. You have a rule where the target is `import`, but that doesn't help make understand how to build `$(IMPORT_DIR)/sf.done`. If you want make to run that recipe to build `$(IMPORT_DIR)/sf.done` then the target must be `$(IMPORT_DIR)/sf.done`, not `import`. – MadScientist May 31 '14 at 14:52
  • Sorry, just discovered your most recent comment! Have you seen my question's UPDATE from the day before your previous comment? I think it implements your advice, but `make` still rebuilds the target, even when it's older than both `transform.R` and `$(IMPORT_DIR)/sf.done`. – Aleksandr Blekh Jun 07 '14 at 05:29
  • 1
    There is nothing in your makefile that creates the `import.done` file. Thus it will never be up to date and the recipe will always be run. I suppose you're missing the `touch $@` command on the `import.done` target, like you have on the other targets. – MadScientist Jun 07 '14 at 11:16
  • Just discovered your previous comment! There is no `touch` command, because it's a part of the build process, called by the rule. File `import.done` is located in the `import` directory and is a part of that build workflow (and, essentially, `import/Makefile` is responsible and has rule for creating this file, when needed). Thus, I can't just simply create this file as a part of `prepare` build workflow. Why does `prepare/Makefile` need to have a rule for creating a file (dependency), external to this subsystem? It just needs to check the existence of the file and compare timestamps. – Aleksandr Blekh Jun 22 '14 at 13:04
  • I just realized that you were probably talking about creating local `prepare/import.done` file, not the external `import/import.done`. Then, yes, `touch` is needed. Sorry about the confusion. UPDATE: Just tried the changes, but still getting `No rule to make target '../prepare/transform.done', needed by 'prepare.done'`. Why does it want to make EXTERNAL target (meaning that its rule is located in the external `Makefile`) rather than just to compare timestamps? – Aleksandr Blekh Jun 22 '14 at 13:17
  • 1
    Sorry, but based on the makefile in the example above there's no `prepare.done` target, much less one that depends on `../prepare/transform.done`, so I can't say anything about your problem other than what I said in my first comment above: make treats every prerequisite as if it were a target, and tries to build it. If the file exists and make doesn't know how to build it, then make will assume it's a "source file" that can't be rebuilt. If the file doesn't exist and make doesn't know how to build it, then you get the error message you show here. – MadScientist Jun 23 '14 at 14:21
  • Thank you and I'm sorry for being so persistent. But, it's still unclear to me how to correctly handle the situation when a dependency (prerequisite) in one `Makefile` (one directory) is a target in another `Makefile` (with build rule specified here; another directory). I haven't found any info on multi-directory builds with chain dependencies (collect -> prepare -> analyze -> report). – Aleksandr Blekh Jun 24 '14 at 03:21
  • 1
    At this point I think your question has diverged too far, and the examples provided in the question are not relevant enough, to continue the discussion in comments here. I recommend you open a new question about this aspect (how to deal with recursive builds like this). – MadScientist Jun 27 '14 at 15:01