16

This is sort of a continuation of question from here. The problem is that there is a rule generating multiple outputs from a single input, and the command is time-consuming so we would prefer to avoid recomputation. Now there is an additional twist, that we want to keep files from being deleted as intermediate files, and rules involve wildcards to allow for parameters.

The solution suggested was that we set up the following rule:

file-a.out: program file.in
    ./program file.in file-a.out file-b.out file-c.out

file-b.out: file-a.out
    @

file-c.out: file-b.out
    @

Then, calling make file-c.out creates both and we avoid issues with running make in parallel with -j switch. All fine so far.

The problem is the following. Because the above solution sets up a chain in the DAG, make considers it differently; the files file-a.out and file-b.out are treated as intermediate files, and they by default get deleted as unnecessary as soon as file-c.out is ready.

A way of avoiding that was mentioned somewhere here, and consists of adding file-a.out and file-b.out as dependencies of a target .SECONDARY, which keeps them from being deleted. Unfortunately, this does not solve my case because my rules use wildcard patters; specifically, my rules look more like this:

file-a-%.out: program file.in
    ./program $* file.in file-a-$*.out file-b-$*.out file-c-$*.out

file-b-%.out: file-a-%.out
    @

file-c-%.out: file-b-%.out
    @

so that one can pass a parameter that gets included in the file name, for example by running

make file-c-12.out

The solution that make documentation suggests is to add these as implicit rules to the list of dependencies of .PRECIOUS, thus keeping these files from being deleted.

The solution with .PRECIOUS works, but it also prevents these files from being deleted when a rule fails and files are incomplete. Is there any other way to make this work?

A hack to solve this is to define a target .SECONDARY with no prerequisites, i.e.,

.SECONDARY:

which informs make that all files should be treated as secondary and thus not removed, unless when make gets interrupted or a rule fails. Unfortunately, this does not allow for selecting a subset of rules with wildcards to work this way, so I consider this only a hack (even though it's useful).

Community
  • 1
  • 1
makesaurus
  • 1,393
  • 3
  • 10
  • 7
  • Hmm, there's no stem in the prerequisites there (i.e. you're using file.in rather than file-%.in). It becomes a lot easier if you build file-$a-$n.out from file-$n.in, and it's weird the way it is (make file-c-17.out file-c-18.out) will produce identical outputs with different names unless there is some hidden input source like time. Could the rule be changed to file-a-%.out: program file-%.in ? – p00ya Jun 15 '10 at 15:41
  • It's irrelevant to the question at hand. These are just made up rules, the point being that they contain wildcards. The real rules are significantly more complicated but that complexity would hide the problem I am facing. – makesaurus Jun 15 '10 at 15:44
  • See my answer for why it is relevant. – p00ya Jun 15 '10 at 15:56
  • No, because the fact that `file.in` does not depend on `$*` comes from somewhere; there is a reason for that. For example, `file.in` keeps all the constant parameters while % is used for something I would like to vary. – makesaurus Jun 15 '10 at 16:03
  • Instead of a chain, have you tried a secondary join. See this answer: http://stackoverflow.com/questions/2973445/gnu-makefile-rule-generating-a-few-targets-from-a-single-source-file/4642560#4642560 – ctrl-alt-delor Jan 10 '11 at 21:29
  • > "*`SECONDARY` ... informs make that all files should be treated as secondary and thus not removed, unless when make gets interrupted or a rule fails.*" The files marked `SECONDARY` are not removed *even if* make is interrupted. – Kaz Dec 05 '14 at 00:34
  • > "*Unfortunately, [`.SECONDARY:`] does not allow for selecting a subset of rules with wildcards to work this way, so I consider this only a hack*." Though it doesn't allow patterns, you can give it a list of specific targets which are to be marked. This is acceptable if you can generate these lists, e.g. `INTERMEDIATES=$(OBJS:.o=.i)` and then `.SECONDARY: $(INTERMEDIATES)`. – Kaz Dec 05 '14 at 00:36

2 Answers2

26

The Simplest Thing

file-a-%.out file-b-%.out file-c-%.out: program file.in
    ./program $* file.in file-a-$*.out file-b-$*.out file-c-$*.out

will do exactly what you want.

(Pattern rules with multiple targets are different than the normal rule with multiple targets that you were asking about here. See the bison example in the make manual.)

Community
  • 1
  • 1
slowdog
  • 6,076
  • 2
  • 27
  • 30
2

If, instead of a single file.in, your outputs were generated from a prerequisite file that included the stem, i.e.

file-a.out: program file-%.in
    ./program file-$*.in file-a-$*.out file-b-$*.out file-c-$*.out

you can then build a list of all possible target matches:

inputs = $(wildcard file-*.in)
secondaries = $(patsubst file-%.in,file-a-%.out,$(inputs)) \
    $(patsubst file-%.in,file-b-%.out,$(inputs))

Similarly if the stem comes from a finite set:

batchnos = 17 18 19 20 
batchnos = $(shell seq 17 20)
secondaries = $(patsubst %,file-a-%.out,$(batchnos)) $(patsubst %,file-b-%.out,$(batchnos))

Then just add these as prereqs to the .SECONDARY target

.SECONDARY: $(secondaries)
p00ya
  • 3,659
  • 19
  • 17
  • That is a possible solution that is not always very practical. In my case it was simpler to leave `.SECONDARY: ` instead of listing all possibilities. Also, it doesn't require putting `%` in the name of the input file. All you need is to add all possible names of output files to the list of secondaries. – makesaurus Jun 15 '10 at 16:01